517
76%

Ran 05 Jun 2018 07:24AM UTC

Jobs 1

Files 1013

Run time 3min

Badge

Embed ▾

pending completion

Build # 517

Build Type

push

jenkins

Committed by manishgupta88

Commit Message

[CARBONDATA-2571] Calculating the carbonindex and carbondata file size of a table is wrong

Problem:
While calculating the carbonindex files size, we are checking either index file or merge file. But in PR#2333, implementation is changed to fill both
the file name and the merge file name. So, we have to consider both fields.

Solution:
While calculating the carbonindex files size, we have to consider both the files and mergeFileName fields. We should get the list of index files from
these 2 fields and then calculate the size of the files.

This closes #2358

Run Details

17 of 17 new or added lines in 1 file covered. (100.0%)

50089 of 69495 relevant lines covered (72.08%)

0.97 hits per line

New Missed Lines in Diff

Lines	Coverage	∆	File
3	82.12	0.12%	core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java

Uncovered Existing Lines

Lines	Coverage	∆	File
1	87.18	-2.56%	core/src/main/java/org/apache/carbondata/core/scan/complextypes/StructQueryType.java
1	91.81	-0.34%	hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java
1	98.32	-0.34%	core/src/main/java/org/apache/carbondata/core/datastore/block/SegmentProperties.java
1	69.35	-0.5%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonAlterTableCompactionCommand.scala
1	86.76	-1.47%	integration/spark-common/src/main/java/org/apache/carbondata/spark/util/SparkDataTypeConverterImpl.java
1	51.16	-1.16%	core/src/main/java/org/apache/carbondata/core/datastore/page/LazyColumnPage.java
1	84.71	-1.18%	core/src/main/java/org/apache/carbondata/core/metadata/schema/table/DataMapSchema.java
1	96.67	-1.11%	integration/spark2/src/main/spark2.2/org/apache/spark/sql/hive/CarbonSessionState.scala
1	91.3	-4.35%	core/src/main/java/org/apache/carbondata/core/scan/expression/logical/RangeExpression.java
1	94.12	-2.94%	core/src/main/java/org/apache/carbondata/core/metadata/schema/table/column/CarbonColumn.java
1	93.1	-3.45%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonDeleteLoadByLoadDateCommand.scala
1	90.2	-1.96%	integration/spark2/src/main/scala/org/apache/spark/sql/CarbonDataFrameWriter.scala
1	17.39	-4.35%	core/src/main/java/org/apache/carbondata/core/scan/expression/conditional/StartsWithExpression.java
1	75.58	1.16%	core/src/main/java/org/apache/carbondata/core/util/SessionParams.java
1	93.96	-0.67%	integration/spark-common/src/main/scala/org/apache/spark/sql/profiler/ProfilerLogger.scala
1	95.85	-0.26%	processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/sort/TimSort.java
1	75.47	-1.89%	core/src/main/java/org/apache/carbondata/core/scan/complextypes/PrimitiveQueryType.java
1	73.62	-0.61%	core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
1	66.67	-16.67%	core/src/main/java/org/apache/carbondata/core/scan/expression/logical/TrueExpression.java
1	83.87	-1.61%	integration/spark2/src/main/scala/org/apache/spark/sql/hive/execution/command/CarbonHiveCommands.scala
1	94.44	-5.56%	processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/comparator/UnsafeRowComparatorForNormalDims.java
1	77.59	-1.72%	core/src/main/java/org/apache/carbondata/core/datamap/DistributableDataMapFormat.java
1	92.86	-7.14%	core/src/main/java/org/apache/carbondata/core/datamap/dev/DataMapFactory.java
1	84.85	-3.03%	core/src/main/java/org/apache/carbondata/core/scan/complextypes/ArrayQueryType.java
1	87.5	-12.5%	core/src/main/java/org/apache/carbondata/core/enums/EscapeSequences.java
1	92.86	-3.57%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonDeleteLoadByIdCommand.scala
1	95.68	-0.62%	processing/src/main/java/org/apache/carbondata/processing/store/TablePage.java
1	90.57	-1.89%	core/src/main/java/org/apache/carbondata/core/datastore/chunk/impl/FixedLengthDimensionColumnPage.java
1	86.49	-0.68%	core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/DefaultEncodingFactory.java
1	84.48	-1.72%	core/src/main/java/org/apache/carbondata/core/writer/sortindex/CarbonDictionarySortIndexWriterImpl.java
1	75.0	-8.33%	processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortScopeOptions.java
1	81.38	-0.69%	processing/src/main/java/org/apache/carbondata/processing/loading/sort/unsafe/UnsafeSortDataRows.java
1	78.76	-0.88%	core/src/main/java/org/apache/carbondata/core/indexstore/BlockletDetailInfo.java
1	94.21	-0.41%	processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerModel.java
1	54.93	-1.41%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonCleanFilesCommand.scala
1	99.15	-0.85%	processing/src/main/java/org/apache/carbondata/processing/loading/CarbonDataLoadConfiguration.java
1	91.67	-8.33%	core/src/main/java/org/apache/carbondata/core/datastore/chunk/impl/AbstractDimensionColumnPage.java
2	78.98	-0.64%	core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java
2	82.12	0.12%	core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java
2	87.07	-1.36%	integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonAnalysisRules.scala
2	68.97	-6.9%	core/src/main/java/org/apache/carbondata/core/datamap/DataMapRegistry.java
2	96.04	-1.98%	processing/src/main/java/org/apache/carbondata/processing/loading/model/LoadOption.java
2	33.33	-16.67%	core/src/main/java/org/apache/carbondata/core/scan/expression/exception/FilterUnsupportedException.java
2	86.79	-1.89%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableDataTypeChangeCommand.scala
2	92.22	-1.2%	processing/src/main/java/org/apache/carbondata/processing/loading/model/CarbonLoadModelBuilder.java
2	52.6	-0.35%	processing/src/main/java/org/apache/carbondata/processing/merger/CarbonDataMergerUtil.java
2	85.48	-3.23%	integration/spark-common/src/main/scala/org/apache/spark/sql/catalyst/AbstractCarbonSparkSQLParser.scala
2	84.62	-7.69%	core/src/main/java/org/apache/carbondata/core/datamap/dev/expr/DataMapExprWrapperImpl.java
2	0.0	-50.0%	common/src/main/java/org/apache/carbondata/common/exceptions/MetadataProcessException.java
2	54.55	-9.09%	integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonMetaStore.scala
2	77.5	-5.0%	core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/unsafe/UnsafeAbstractDimensionDataChunkStore.java
2	90.91	-2.02%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonDropTableCommand.scala
2	51.14	-2.27%	core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/adaptive/AdaptiveIntegralCodec.java
2	89.13	-1.45%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateTableHelper.scala
2	78.0	-2.0%	core/src/main/java/org/apache/carbondata/core/readcommitter/LatestFilesReadCommittedScope.java
2	94.07	-1.69%	processing/src/main/java/org/apache/carbondata/processing/loading/converter/impl/RowConverterImpl.java
2	71.43	-7.14%	core/src/main/java/org/apache/carbondata/core/locks/AbstractCarbonLock.java
2	82.67	-1.33%	processing/src/main/java/org/apache/carbondata/processing/loading/sort/impl/UnsafeBatchParallelReadMergeSorterImpl.java
2	95.15	-1.94%	core/src/main/java/org/apache/carbondata/core/metadata/schema/table/DiskBasedDMSchemaStorageProvider.java
2	97.18	-2.82%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableAddColumnCommand.scala
2	82.72	-2.47%	processing/src/main/java/org/apache/carbondata/processing/datatypes/ArrayDataType.java
3	83.59	-2.34%	processing/src/main/java/org/apache/carbondata/processing/merger/RowResultMergerProcessor.java
3	85.96	-1.75%	integration/spark2/src/main/scala/org/apache/spark/util/AlterTableUtil.scala
3	90.91	-5.45%	processing/src/main/java/org/apache/carbondata/processing/loading/converter/impl/NonDictionaryFieldConverterImpl.java
3	76.12	-3.57%	store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReaderBuilder.java
3	84.47	-2.91%	processing/src/main/java/org/apache/carbondata/processing/datatypes/StructDataType.java
3	83.9	-1.46%	integration/spark2/src/main/scala/org/apache/spark/sql/CarbonSession.scala
3	95.91	-1.75%	processing/src/main/java/org/apache/carbondata/processing/loading/DataLoadProcessBuilder.java
3	86.96	-2.61%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/datamap/CarbonDropDataMapCommand.scala
3	55.48	-0.97%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelFilterExecuterImpl.java
3	85.41	-1.07%	core/src/main/java/org/apache/carbondata/core/scan/executor/impl/AbstractQueryExecutor.java
3	76.92	-0.62%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateUtil.scala
3	78.06	-1.27%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeGrtThanFiterExecuterImpl.java
3	79.1	-4.48%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/datamap/CarbonCreateDataMapCommand.scala
3	81.13	-5.66%	core/src/main/java/org/apache/carbondata/core/locks/CarbonLockUtil.java
3	84.0	-6.0%	integration/spark2/src/main/java/org/apache/carbondata/datamap/IndexDataMapProvider.java
3	72.73	-27.27%	processing/src/main/java/org/apache/carbondata/processing/loading/exception/CarbonDataLoadingException.java
3	92.31	-7.69%	processing/src/main/java/org/apache/carbondata/processing/loading/converter/impl/DirectDictionaryFieldConverterImpl.java
3	77.53	-3.37%	processing/src/main/java/org/apache/carbondata/processing/loading/BadRecordsLogger.java
4	78.26	-17.39%	core/src/main/java/org/apache/carbondata/core/indexstore/PartitionSpec.java
4	85.42	-2.78%	core/src/main/java/org/apache/carbondata/core/metadata/schema/table/AggregationDataMapSchema.java
4	86.03	-1.12%	integration/spark2/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala
4	93.18	-4.55%	core/src/main/java/org/apache/carbondata/core/scan/collector/impl/DictionaryBasedResultCollector.java
4	86.67	-2.96%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/mutation/CarbonProjectForUpdateCommand.scala
4	88.35	-3.88%	core/src/main/java/org/apache/carbondata/core/scan/processor/DataBlockIterator.java
4	87.5	1.29%	store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReader.java
4	70.0	-13.33%	processing/src/main/java/org/apache/carbondata/processing/loading/DataLoadExecutor.java
4	76.03	-3.31%	integration/spark2/src/main/java/org/apache/carbondata/spark/vectorreader/VectorizedCarbonRecordReader.java
5	80.78	-1.02%	processing/src/main/java/org/apache/carbondata/processing/util/CarbonLoaderUtil.java
5	0.0	-100.0%	core/src/main/java/org/apache/carbondata/core/exception/ConcurrentOperationException.java
5	0.0	-41.67%	core/src/main/java/org/apache/carbondata/core/scan/executor/exception/QueryExecutionException.java
5	0.0	-71.43%	hadoop/src/main/java/org/apache/carbondata/hadoop/CacheClient.java
5	72.09	-1.1%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateListeners.scala
5	0.0	-62.5%	integration/spark2/src/main/scala/org/apache/spark/sql/CustomDeterministicExpression.scala
6	90.96	-1.2%	integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
6	88.68	-11.32%	core/src/main/java/org/apache/carbondata/core/preagg/AggregateTableSelector.java
6	73.33	-20.0%	integration/spark2/src/main/spark2.2/org/apache/spark/sql/hive/CarbonSessionUtil.scala
6	70.1	-0.9%	integration/spark2/src/main/scala/org/apache/carbondata/spark/rdd/CarbonDataRDDFactory.scala
6	75.41	-9.84%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/mutation/CarbonProjectForDeleteCommand.scala
7	64.38	-9.59%	integration/spark-common/src/main/scala/org/apache/spark/sql/test/util/QueryTest.scala
7	0.0	-29.17%	hadoop/src/main/java/org/apache/carbondata/hadoop/CacheAccessClient.java
7	29.41	-20.59%	datamap/lucene/src/main/java/org/apache/carbondata/datamap/lucene/LuceneFineGrainDataMapFactory.java
7	78.67	-4.67%	integration/spark-common/src/main/scala/org/apache/carbondata/api/CarbonStore.scala
8	67.96	-2.59%	core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java
8	52.31	-12.31%	integration/spark2/src/main/scala/org/apache/spark/util/SparkTypeConverter.scala
8	76.34	-3.57%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/ExcludeFilterExecuterImpl.java
8	79.82	-7.02%	datamap/lucene/src/main/java/org/apache/carbondata/datamap/lucene/LuceneDataMapFactoryBase.java
8	0.0	-80.0%	processing/src/main/java/org/apache/carbondata/processing/loading/dictionary/DirectDictionary.java
8	75.11	-2.11%	core/src/main/java/org/apache/carbondata/core/datamap/DataMapStoreManager.java
8	59.32	-13.56%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/AndFilterExecuterImpl.java
9	68.08	-2.54%	core/src/main/java/org/apache/carbondata/core/scan/executor/util/QueryUtil.java
9	63.59	-4.89%	datamap/lucene/src/main/java/org/apache/carbondata/datamap/lucene/LuceneFineGrainDataMap.java
9	72.84	-11.11%	hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonFileInputFormat.java
9	92.48	-2.51%	core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java
9	73.16	-4.74%	integration/spark2/src/main/scala/org/apache/spark/sql/CarbonSource.scala
9	66.09	-1.74%	core/src/main/java/org/apache/carbondata/core/util/DataTypeUtil.java
9	90.0	-10.0%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableDropColumnCommand.scala
9	75.22	-3.91%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeGrtrThanEquaToFilterExecuterImpl.java
10	72.73	-6.06%	core/src/main/java/org/apache/carbondata/core/datamap/DataMapChooser.java
10	84.62	-10.99%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonCreateTableCommand.scala
10	80.28	-1.08%	integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonPreAggregateRules.scala
10	0.0	-100.0%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableUnsetCommand.scala
11	72.44	-4.33%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanFilterExecuterImpl.java
12	71.43	-1.88%	core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
12	71.32	-9.3%	core/src/main/java/org/apache/carbondata/core/datastore/page/UnsafeDecimalColumnPage.java
12	88.53	-2.99%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/strategy/CarbonLateDecodeStrategy.scala
14	89.36	-9.93%	integration/spark-common/src/main/scala/org/apache/spark/util/PartitionUtils.scala
15	75.0	-9.87%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessorStepOnSpark.scala
15	87.74	-7.08%	core/src/main/java/org/apache/carbondata/core/scan/result/vector/MeasureDataVectorProcessor.java
15	79.16	-2.74%	integration/spark2/src/main/scala/org/apache/spark/sql/optimizer/CarbonLateDecodeRule.scala
15	84.26	-13.89%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonDescribeFormattedCommand.scala
15	78.09	-8.43%	integration/spark2/src/main/scala/org/apache/spark/sql/parser/CarbonSparkSqlParser.scala
15	80.85	-10.64%	hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonOutputCommitter.java
15	79.35	-3.07%	core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentStatusManager.java
16	75.69	-3.67%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/GlobalDictionaryUtil.scala
17	67.27	-29.09%	core/src/main/java/org/apache/carbondata/core/preagg/TimeSeriesUDF.java
18	71.43	-7.35%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/strategy/DDLStrategy.scala
18	81.87	-10.53%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableRenameCommand.scala
20	83.76	-8.55%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/SparkCarbonTableFormat.scala
21	77.16	-2.92%	integration/spark-common/src/main/scala/org/apache/spark/sql/catalyst/CarbonDDLSqlParser.scala
21	68.68	-7.92%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RangeValueFilterExecuterImpl.java
22	59.41	-4.6%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/CommonUtil.scala
23	49.69	-7.14%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/CarbonScalaUtil.scala
23	79.93	-7.96%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonGlobalDictionaryRDD.scala
25	56.48	-23.15%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/RefreshCarbonTableCommand.scala
31	57.58	-31.31%	store/search/src/main/java/org/apache/carbondata/store/worker/SearchRequestHandler.java
31	69.32	-8.81%	integration/spark2/src/main/scala/org/apache/spark/sql/optimizer/CarbonFilters.scala
32	0.0	-66.67%	integration/spark2/src/main/spark2.2/org/apache/spark/sql/hive/CreateCarbonSourceTableAsSelectCommand.scala
34	67.73	-10.54%	integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
36	58.76	-18.56%	processing/src/main/java/org/apache/carbondata/processing/datatypes/PrimitiveDataType.java
36	74.88	-8.29%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonScanRDD.scala
41	0.0	-73.21%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonCreateTableAsSelectCommand.scala
46	85.15	-10.04%	core/src/main/java/org/apache/carbondata/core/metadata/SegmentFileStore.java
48	85.52	-7.32%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
59	0.0	-72.84%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/partition/CarbonAlterTableAddHivePartitionCommand.scala
84	14.22	-38.53%	datamap/lucene/src/main/java/org/apache/carbondata/datamap/lucene/LuceneDataMapWriter.java
112	2.31	-86.15%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/datasources/SparkCarbonFileFormat.scala

Jobs

ID	Job ID	Ran	Files	Coverage
1	517.1	05 Jun 2018 07:24AM UTC	1013	72.08

apache / carbondata / 517
76%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

New Missed Lines in Diff

Uncovered Existing Lines

Jobs

Source Files on build 517

apache / carbondata / 517 76%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

New Missed Lines in Diff

Uncovered Existing Lines

Jobs

Source Files on build 517

apache / carbondata / 517
76%

README BADGES
x