421
76%

Ran 10 May 2018 04:58PM UTC

Jobs 1

Files 1017

Run time 3min

Badge

Embed ▾

pending completion

Build # 421

Build Type

push

jenkins

Committed by ravipesala

Commit Message

[CARBONDATA-2443][SDK] Multi level complex type support for AVRO based SDK

Problem:
Problem inferring the complex type schema with boolean array type from the store created using SDK writer

Analysis:
When we create an external table and infer the schema from store created using SDK writer, the operation fails because of complex type field with boolean array dataType. This is because during schema creation by SDK writer, for array type children a child with column name val is added.
While parsing the logic to append the parent name with child column name is missing for boolean type which is causing this problem.

Solution:
Handle the parsing for boolean type

This closes #2294

Run Details

18 of 18 new or added lines in 3 files covered. (100.0%)

49414 of 68801 relevant lines covered (71.82%)

0.95 hits per line

New Missed Lines in Diff

Lines	Coverage	∆	File
2	80.78	-0.43%	integration/spark-common/src/main/scala/org/apache/spark/sql/catalyst/CarbonDDLSqlParser.scala

Uncovered Existing Lines

Lines	Coverage	∆	File
1	72.73	-9.09%	store/sdk/src/main/java/org/apache/carbondata/store/MetaCachedCarbonStore.java
1	81.46	-0.49%	integration/spark2/src/main/scala/org/apache/spark/sql/CarbonSession.scala
1	42.13	-0.39%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/CastExpressionOptimization.scala
1	91.04	-1.49%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/datamap/CarbonCreateDataMapCommand.scala
1	77.27	-4.55%	core/src/main/java/org/apache/carbondata/core/scan/filter/resolver/resolverinfo/visitor/FilterInfoTypeVisitorFactory.java
1	87.5	-12.5%	core/src/main/java/org/apache/carbondata/core/scan/filter/intf/RowImpl.java
1	93.86	-0.26%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/strategy/CarbonLateDecodeStrategy.scala
1	92.31	-7.69%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/TrueFilterExecutor.java
1	66.67	-11.11%	core/src/main/java/org/apache/carbondata/core/statusmanager/FileFormat.java
2	92.31	-7.69%	core/src/main/java/org/apache/carbondata/core/dictionary/client/NonSecureDictionaryClient.java
2	86.59	-0.56%	integration/spark2/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala
2	72.93	-0.32%	core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
2	50.0	-8.33%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/KeyVal.scala
2	95.24	-4.76%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/strategy/StreamingTableStrategy.scala
2	88.89	-11.11%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/OrFilterExecuterImpl.java
2	94.83	-0.22%	integration/spark2/src/main/scala/org/apache/spark/sql/hive/CarbonPreAggregateRules.scala
2	29.13	-1.94%	core/src/main/java/org/apache/carbondata/core/datastore/compression/SnappyCompressor.java
2	90.0	-1.0%	core/src/main/java/org/apache/carbondata/core/scan/filter/FilterExpressionProcessor.java
2	91.92	-2.02%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/table/CarbonDropTableCommand.scala
2	78.26	-8.7%	core/src/main/java/org/apache/carbondata/core/datastore/row/CarbonRow.java
2	73.49	-0.8%	core/src/main/java/org/apache/carbondata/core/util/ByteUtil.java
2	85.96	-1.12%	integration/spark2/src/main/scala/org/apache/spark/sql/parser/CarbonSparkSqlParser.scala
2	81.97	-1.64%	core/src/main/java/org/apache/carbondata/core/scan/model/QueryModel.java
2	69.49	-3.39%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/AndFilterExecuterImpl.java
3	74.77	-2.7%	core/src/main/java/org/apache/carbondata/core/statusmanager/LoadMetadataDetails.java
3	85.06	-3.45%	hadoop/src/main/java/org/apache/carbondata/hadoop/util/CarbonInputFormatUtil.java
3	80.78	-0.43%	integration/spark-common/src/main/scala/org/apache/spark/sql/catalyst/CarbonDDLSqlParser.scala
3	80.65	-3.23%	core/src/main/java/org/apache/carbondata/core/util/comparator/Comparator.java
4	91.07	-0.79%	integration/spark-common/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchemaCommon.scala
4	88.35	-3.88%	core/src/main/java/org/apache/carbondata/core/scan/processor/DataBlockIterator.java
4	67.86	-0.8%	core/src/main/java/org/apache/carbondata/core/util/DataTypeUtil.java
4	71.88	-2.5%	core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
4	0.0	-100.0%	streaming/src/main/java/org/apache/carbondata/streaming/CarbonStreamException.java
5	74.6	-1.98%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanFilterExecuterImpl.java
5	70.0	-4.55%	core/src/main/java/org/apache/carbondata/core/scan/filter/resolver/ConditionalFilterResolverImpl.java
5	58.16	-3.55%	core/src/main/java/org/apache/carbondata/core/datastore/impl/FileFactory.java
6	80.34	-0.46%	core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java
6	0.0	-40.0%	streaming/src/main/java/org/apache/carbondata/streaming/CarbonStreamOutputFormat.java
6	0.0	-100.0%	streaming/src/main/java/org/apache/carbondata/streaming/parser/CSVStreamParserImp.java
7	72.67	-4.07%	processing/src/main/java/org/apache/carbondata/processing/merger/CompactionResultSortProcessor.java
7	81.19	-1.43%	core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentStatusManager.java
7	75.0	-25.0%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonShowLoadsCommand.scala
8	77.87	-3.28%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/strategy/DDLStrategy.scala
8	75.74	-3.4%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeGrtThanFiterExecuterImpl.java
8	89.77	-1.22%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala
8	76.75	-3.51%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeGrtrThanEquaToFilterExecuterImpl.java
9	70.0	-22.5%	streaming/src/main/scala/org/apache/carbondata/streaming/parser/FieldConverter.scala
9	70.59	-13.24%	hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonMultiBlockSplit.java
9	73.79	-3.63%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelRangeLessThanEqualFilterExecuterImpl.java
9	0.0	-100.0%	store/sdk/src/main/java/org/apache/carbondata/store/CarbonRowReadSupport.java
10	67.74	-32.26%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/mutation/IUDCommonUtil.scala
10	75.65	-3.69%	processing/src/main/java/org/apache/carbondata/processing/util/CarbonDataProcessorUtil.java
10	77.66	-5.08%	hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonInputSplit.java
11	84.99	-1.92%	processing/src/main/java/org/apache/carbondata/processing/merger/CarbonDataMergerUtil.java
11	73.03	-12.36%	core/src/main/java/org/apache/carbondata/core/util/DeleteLoadFolders.java
12	0.0	-80.0%	integration/spark2/src/main/scala/org/apache/spark/util/DeleteSegmentByDate.scala
13	0.0	-81.25%	integration/spark2/src/main/scala/org/apache/spark/util/DeleteSegmentById.scala
15	76.61	-3.44%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/util/GlobalDictionaryUtil.scala
15	0.0	-68.18%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonAlterTableFinishStreaming.scala
15	91.6	-4.2%	core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java
15	70.53	-7.89%	integration/spark2/src/main/scala/org/apache/spark/sql/CarbonSource.scala
15	0.0	-71.43%	integration/spark2/src/main/scala/org/apache/spark/util/TableAPIUtil.scala
15	86.55	-3.36%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateListeners.scala
16	68.05	-6.02%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/IncludeFilterExecuterImpl.java
17	14.29	-80.95%	integration/spark-common/src/main/scala/org/apache/carbondata/streaming/CarbonStreamingQueryListener.scala
19	0.0	-67.86%	core/src/main/java/org/apache/carbondata/core/scan/filter/resolver/resolverinfo/visitor/RangeDirectDictionaryVisitor.java
20	69.06	-7.55%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RangeValueFilterExecuterImpl.java
21	57.87	-2.54%	core/src/main/java/org/apache/carbondata/core/scan/filter/FilterUtil.java
22	0.0	-88.0%	integration/spark2/src/main/scala/org/apache/spark/util/CleanFiles.scala
23	53.7	-7.4%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelFilterExecuterImpl.java
23	68.0	-15.33%	integration/spark-common/src/main/scala/org/apache/carbondata/api/CarbonStore.scala
25	77.19	-5.76%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonScanRDD.scala
26	0.0	-100.0%	streaming/src/main/scala/org/apache/carbondata/streaming/parser/RowStreamParserImp.scala
28	67.41	-12.5%	core/src/main/java/org/apache/carbondata/core/scan/filter/executer/ExcludeFilterExecuterImpl.java
29	52.78	-26.85%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/RefreshCarbonTableCommand.scala
31	77.16	-10.73%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonGlobalDictionaryRDD.scala
32	0.0	-71.11%	streaming/src/main/java/org/apache/carbondata/streaming/CarbonStreamInputFormat.java
33	43.48	-47.83%	integration/spark2/src/main/scala/org/apache/carbondata/store/SparkCarbonStore.scala
34	63.32	-17.09%	integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonAlterTableCompactionCommand.scala
43	78.42	-14.04%	hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java
43	25.0	-63.24%	integration/spark-common/src/main/java/org/apache/carbondata/spark/util/SparkDataTypeConverterImpl.java
67	0.0	-90.54%	streaming/src/main/java/org/apache/carbondata/streaming/StreamBlockletWriter.java
102	0.0	-90.27%	streaming/src/main/java/org/apache/carbondata/streaming/StreamBlockletReader.java
119	0.0	-83.8%	integration/spark-common/src/main/scala/org/apache/carbondata/streaming/StreamSinkFactory.scala
134	0.0	-88.74%	streaming/src/main/java/org/apache/carbondata/streaming/CarbonStreamRecordWriter.java
166	0.0	-93.26%	integration/spark-common/src/main/scala/org/apache/spark/sql/execution/streaming/CarbonAppendableStreamSink.scala
188	0.0	-86.24%	integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/StreamHandoffRDD.scala
192	0.0	-86.1%	streaming/src/main/java/org/apache/carbondata/streaming/segment/StreamSegment.java
325	0.0	-82.28%	streaming/src/main/java/org/apache/carbondata/streaming/CarbonStreamRecordReader.java

Jobs

ID	Job ID	Ran	Files	Coverage
1	421.1	10 May 2018 04:58PM UTC	1017	71.82

apache / carbondata / 421
76%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

New Missed Lines in Diff

Uncovered Existing Lines

Jobs

Source Files on build 421

apache / carbondata / 421 76%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

New Missed Lines in Diff

Uncovered Existing Lines

Jobs

Source Files on build 421

apache / carbondata / 421
76%

README BADGES
x