taosdata / TDengine / #3660
73%
main: 60%

LAST BUILD BRANCH: 3.0
DEFAULT BRANCH: main
Ran
Jobs 1
Files 520
Run time 3min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 62.039% (-1.3%) from 63.314%
#3660

push

travis-ci

web-flow 
feat(stream): support stream processing for virtual tables (#30144)

* enh: add client processing

* enh: add mnode vtables processing

* enh: add mnode vtable processing

* enh: add normal child vtable support

* fix: compile issues

* fix: compile issues

* fix: create stream issues

* fix: multi stream scan issue

* fix: remove debug info

* fix: agg task and task level issues

* fix: correct task output type

* fix: split vtablescan from agg

* fix: memory leak issues

* fix: add limitations

* Update 09-error-code.md

* Update 09-error-code.md

* fix: remove usless case

* feat(stream): extract original table data in source scan task

Implemented functionality in the source task to extract data
corresponding to the virtual table from the original table using WAL.
The extracted data is then sent to the downstream merge task for further
processing.

* feat(stream): multi-way merge using loser tree in virtual merge task

Implemented multi-way merge in the merge task using a loser tree to
combine data from multiple original table into a single virtual table.
The merged virtual table data is then pushed downstream for further
processing.  Introduced memory limit handling during the merge process
with configurable behavior when the memory limit is reached.

* fix(test): remove useless cases

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: Pan Wei <72057773+dapan1121@users.noreply.github.com>

154078 of 317582 branches covered (48.52%)

Branch coverage included in aggregate %.

313 of 2391 new or added lines in 34 files covered. (13.09%)

26134 existing lines in 205 files now uncovered.

240261 of 318051 relevant lines covered (75.54%)

16655189.27 hits per line

Uncovered Changes

Lines Coverage ∆ File
432
44.38
 -32.22% source/dnode/mnode/impl/src/mndScheduler.c
416
0.0
 source/libs/executor/src/virtualtablescanoperator.c
352
48.47
 -16.41% source/dnode/vnode/src/tq/tqRead.c
156
66.6
 -5.02% source/libs/nodes/src/nodesCodeFuncs.c
121
68.55
 -1.47% source/common/src/msg/tmsg.c
115
60.05
 -8.28% source/libs/stream/src/streamTask.c
103
49.0
 -8.94% source/dnode/vnode/src/vnd/vnodeQuery.c
76
48.62
 -1.14% source/libs/catalog/src/ctgAsync.c
73
62.82
 -7.41% source/libs/stream/src/streamDispatch.c
50
55.33
 -1.01% source/libs/catalog/src/ctgUtil.c
31
63.73
 -1.49% source/libs/executor/src/scanoperator.c
28
71.84
 -4.48% source/libs/stream/src/streamCheckStatus.c
25
50.5
 2.29% source/libs/qcom/src/querymsg.c
24
49.37
 3.73% source/libs/catalog/src/ctgRemote.c
20
81.86
 -0.53% source/libs/nodes/src/nodesUtilFuncs.c
12
47.3
 1.85% source/libs/catalog/src/ctgCache.c
11
75.03
 -0.99% source/libs/parser/src/parTranslater.c
10
80.31
 0.39% source/libs/planner/src/planSpliter.c
9
64.85
 -1.06% source/libs/stream/src/streamCheckpoint.c
4
54.37
 -3.13% source/libs/executor/src/operator.c
3
80.87
 -0.34% source/libs/planner/src/planLogicCreater.c
2
61.14
 0.18% source/dnode/mnode/impl/src/mndStream.c
2
59.77
 -1.29% source/dnode/vnode/src/vnd/vnodeSvr.c
2
72.71
 -0.29% source/libs/parser/src/parUtil.c
1
69.67
 0.33% source/util/src/tlrucache.c

Coverage Regressions

Lines Coverage ∆ File
1907
75.03
 -0.99% source/libs/parser/src/parTranslater.c
1022
68.55
 -1.47% source/common/src/msg/tmsg.c
975
66.6
 -5.02% source/libs/nodes/src/nodesCodeFuncs.c
929
62.52
 0.21% tools/taos-tools/src/taosdump.c
838
63.73
 -1.49% source/libs/executor/src/scanoperator.c
698
66.18
 -0.61% source/libs/scalar/src/filter.c
654
71.67
 -0.72% source/libs/function/src/builtinsimpl.c
629
55.39
 -4.64% source/libs/executor/src/sysscanoperator.c
591
74.35
 -1.42% source/libs/planner/src/planOptimizer.c
537
43.73
 -10.99% source/dnode/vnode/src/meta/metaTable2.c
533
71.35
 0.13% source/libs/executor/src/streamtimewindowoperator.c
531
47.3
 1.85% source/libs/catalog/src/ctgCache.c
521
67.11
 0.4% source/dnode/mnode/impl/src/mndStb.c
503
36.09
 -18.55% source/libs/executor/src/dynqueryctrloperator.c
469
61.63
 1.02% source/dnode/vnode/src/tsdb/tsdbCache.c
451
48.62
 -1.14% source/libs/catalog/src/ctgAsync.c
445
47.46
 -13.96% source/dnode/vnode/src/meta/metaEntry2.c
435
48.41
 -12.11% source/dnode/mnode/impl/src/mndSma.c
421
61.0
 0.16% source/common/src/tdataformat.c
418
61.14
 0.18% source/dnode/mnode/impl/src/mndStream.c
410
64.51
 -1.09% source/common/src/tdatablock.c
399
68.05
 0.01% source/libs/parser/src/parAstCreater.c
399
69.04
 -0.75% source/libs/planner/src/planPhysiCreater.c
378
69.76
 0.02% source/dnode/vnode/src/tsdb/tsdbRead2.c
357
53.52
 -2.78% source/libs/scalar/src/sclfunc.c
338
64.14
 -0.47% source/client/src/clientImpl.c
314
54.79
 -13.77% source/libs/stream/src/streamSessionState.c
306
60.62
 -3.24% source/libs/stream/src/tstreamFileState.c
306
55.21
 -0.8% source/util/src/tcompare.c
295
82.74
 -0.36% source/libs/nodes/src/nodesMsgFuncs.c
286
57.07
 -1.09% source/client/src/clientMain.c
283
52.08
 -0.06% source/libs/stream/src/streamBackendRocksdb.c
268
73.99
 -0.45% source/libs/parser/src/parInsertSql.c
267
56.85
 -1.3% source/libs/command/src/explain.c
238
55.33
 -1.01% source/libs/catalog/src/ctgUtil.c
226
54.16
 -6.0% source/libs/executor/src/exchangeoperator.c
221
67.84
 -0.63% source/libs/scalar/src/scalar.c
212
67.34
 0.04% source/common/src/tglobal.c
210
81.86
 -0.53% source/libs/nodes/src/nodesUtilFuncs.c
199
62.82
 -7.41% source/libs/stream/src/streamDispatch.c
197
27.92
 8.9% source/libs/catalog/src/catalog.c
194
48.89
 -1.57% source/libs/scalar/src/sclvector.c
193
59.77
 -1.29% source/dnode/vnode/src/vnd/vnodeSvr.c
190
56.13
 -0.55% source/dnode/vnode/src/meta/metaQuery.c
183
62.25
 -0.37% source/dnode/vnode/src/tqCommon/tqCommon.c
183
49.0
 -8.94% source/dnode/vnode/src/vnd/vnodeQuery.c
183
80.87
 -0.34% source/libs/planner/src/planLogicCreater.c
182
19.01
 3.41% source/libs/executor/src/streamexecutorInt.c
176
62.42
 -10.84% source/libs/stream/src/streamExec.c
166
59.63
 -6.89% source/dnode/vnode/src/meta/metaTable.c
166
59.95
 0.51% source/libs/stream/src/streamMeta.c
166
60.05
 -8.28% source/libs/stream/src/streamTask.c
153
67.23
 -0.32% source/dnode/mnode/impl/src/mndDb.c
153
74.56
 -1.52% source/libs/executor/src/streamtimesliceoperator.c
152
66.62
 2.96% source/libs/command/src/command.c
151
80.31
 0.39% source/libs/planner/src/planSpliter.c
146
72.71
 -0.29% source/libs/parser/src/parUtil.c
142
0.0
 -54.64% source/dnode/vnode/src/sma/smaTimeRange.c
141
59.96
 0.01% source/client/src/clientRawBlockWrite.c
124
64.96
 -0.04% source/libs/executor/src/executor.c
113
56.84
 -11.51% source/dnode/vnode/src/meta/metaEntry.c
112
73.43
 -1.1% source/libs/nodes/src/nodesCloneFuncs.c
111
64.85
 -1.06% source/libs/stream/src/streamCheckpoint.c
105
54.37
 -3.13% source/libs/executor/src/operator.c
105
79.97
 0.6% source/libs/function/src/builtins.c
103
68.08
 -1.02% source/libs/executor/src/executil.c
92
60.63
 0.05% source/libs/qworker/src/qworker.c
91
76.81
 3.14% tools/shell/src/shellEngine.c
89
62.23
 -1.95% source/dnode/vnode/src/tq/tq.c
89
69.71
 -4.09% source/libs/stream/src/streamState.c
87
34.21
 0.29% source/client/src/clientStmt2.c
81
44.38
 -32.22% source/dnode/mnode/impl/src/mndScheduler.c
81
50.5
 2.29% source/libs/qcom/src/querymsg.c
79
64.5
 -5.64% source/client/src/clientStmt.c
77
77.2
 0.01% source/libs/planner/src/planUtil.c
74
65.86
 0.46% source/libs/function/src/detail/tminmax.c
73
48.47
 -16.41% source/dnode/vnode/src/tq/tqRead.c
68
67.22
 -0.2% source/dnode/vnode/src/tsdb/tsdbUtil.c
68
62.26
 -11.11% source/libs/stream/src/streamSched.c
64
82.78
 -1.31% source/libs/parser/src/parAstParser.c
61
0.0
 -66.67% source/dnode/vnode/src/meta/metaSma.c
60
39.83
 -8.69% include/common/tmsg.h
60
63.96
 0.86% source/libs/executor/src/streamintervalsliceoperator.c
59
62.07
 2.31% source/libs/stream/src/streamStartHistory.c
56
74.48
 -0.47% source/libs/executor/src/streamfilloperator.c
55
64.9
 -4.04% source/libs/executor/src/streameventwindowoperator.c
52
57.65
 0.02% source/libs/executor/src/groupcacheoperator.c
52
61.36
 -1.1% source/libs/function/src/tpercentile.c
46
77.24
 -1.77% source/libs/function/src/functionMgt.c
45
69.16
 0.06% source/dnode/vnode/src/tsdb/tsdbCacheRead.c
45
75.34
 0.61% source/libs/executor/src/mergeoperator.c
44
18.93
 -0.06% source/dnode/mnode/impl/src/mndDump.c
44
77.59
 -0.0% source/libs/stream/src/streamQueue.c
42
63.95
 -9.56% source/libs/stream/src/streamData.c
41
68.91
 0.25% source/libs/executor/src/groupoperator.c
38
68.93
 -0.53% source/libs/parser/src/parInsertUtil.c
37
74.47
 -0.2% source/libs/executor/src/tfill.c
35
74.6
 0.56% source/dnode/vnode/src/tq/tqUtil.c
33
65.55
 -1.51% source/libs/executor/src/executorInt.c
33
68.66
 -0.82% source/libs/executor/src/projectoperator.c
32
63.58
 -0.63% source/dnode/mnode/impl/src/mndIndex.c
29
81.08
 2.53% source/libs/parser/src/parTokenizer.c
29
62.45
 -10.15% source/libs/planner/src/planner.c
26
56.61
 1.6% source/libs/qcom/src/queryUtil.c
23
57.69
 -6.46% source/libs/parser/src/parser.c
22
85.69
 1.28% source/common/src/tcol.c
21
71.37
 0.78% source/dnode/mgmt/node_mgmt/src/dmTransport.c
20
80.0
 3.33% source/libs/function/src/detail/tavgfunction.c
19
62.45
 -1.08% source/dnode/mnode/impl/src/mndTrans.c
19
84.38
 -2.96% utils/tsim/src/simExec.c
18
72.93
 -1.07% source/client/src/clientSml.c
18
80.23
 0.42% source/libs/planner/src/planScaleOut.c
17
65.23
 -4.54% source/libs/stream/src/streamSliceState.c
16
59.11
 -0.37% source/dnode/mnode/impl/src/mndVgroup.c
16
62.98
 -2.05% source/util/src/tcompression.c
15
53.66
 -7.32% source/dnode/vnode/src/sma/smaEnv.c
14
56.71
 -0.44% source/dnode/mnode/impl/src/mndInfoSchema.c
14
71.7
 -1.43% source/dnode/mnode/impl/src/mndStreamUtil.c
14
73.67
 1.88% source/dnode/vnode/src/tq/tqStreamTask.c
13
73.03
 13.49% source/common/src/ttypes.c
13
55.32
 -0.08% source/dnode/mnode/impl/src/mndPerfSchema.c
13
37.8
 -1.46% source/libs/parser/src/parInsertStmt.c
12
58.86
 -1.33% source/dnode/vnode/src/sma/smaRollup.c
12
82.93
 0.36% source/libs/parser/src/parAuthenticator.c
11
66.15
 0.17% source/libs/executor/src/tsort.c
10
77.11
 -0.1% source/libs/executor/src/querytask.c
10
61.05
 -2.26% source/libs/wal/src/walWrite.c
8
59.13
 -0.96% source/libs/scheduler/src/schTask.c
7
56.28
 -0.02% source/libs/transport/src/transCli.c
7
80.24
 -4.03% source/util/src/tdigest.c
6
62.4
 0.99% source/client/src/clientHb.c
6
71.79
 -25.64% source/dnode/mnode/impl/src/mndIndexCom.c
5
59.37
 0.02% source/client/src/clientTmq.c
5
55.26
 -0.62% source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
5
63.63
 -1.63% source/dnode/vnode/src/tsdb/tsdbFSet2.c
5
55.3
 0.31% source/libs/executor/src/dataDispatcher.c
5
86.32
 1.1% source/libs/nodes/src/nodesTraverseFuncs.c
5
59.05
 -5.71% source/libs/qworker/src/qwMem.c
5
64.27
 -0.13% source/libs/tdb/src/db/tdbBtree.c
5
90.95
 -2.83% source/util/src/thashutil.c
4
51.76
 -0.49% source/client/src/clientMonitor.c
4
79.69
 -0.27% source/common/src/ttime.c
4
70.43
 -1.02% source/dnode/vnode/src/tsdb/tsdbIter.c
4
55.81
 0.04% source/libs/transport/src/transSvr.c
4
60.28
 -0.35% source/libs/wal/src/walRead.c
3
61.64
 -0.59% source/dnode/mgmt/mgmt_vnode/src/vmInt.c
3
51.92
 -0.13% source/dnode/vnode/src/tsdb/tsdbFS2.c
3
65.78
 -0.75% source/dnode/vnode/src/vnd/vnodeSync.c
3
49.37
 3.73% source/libs/catalog/src/ctgRemote.c
3
78.59
 -1.22% source/libs/executor/src/eventwindowoperator.c
3
58.74
 0.0% source/libs/qworker/src/qwUtil.c
3
39.66
 0.17% source/libs/scheduler/src/schJob.c
3
64.65
 0.5% source/libs/transport/src/transComm.c
3
84.77
 -0.57% source/os/src/osThread.c
3
66.67
 0.02% tools/taos-tools/src/benchInsert.c
2
87.88
 -2.44% include/common/tcommon.h
2
63.02
 -1.04% source/client/src/clientSmlJson.c
2
56.95
 -0.61% source/dnode/mnode/impl/src/mndStreamTransAct.c
2
69.38
 -0.19% source/dnode/mnode/impl/src/mndSubscribe.c
2
85.43
 -0.36% source/dnode/mnode/sdb/src/sdbHash.c
2
78.97
 -0.26% source/dnode/vnode/src/tsdb/tsdbDataFileRW.c
2
70.86
 0.41% source/dnode/vnode/src/tsdb/tsdbReadUtil.c
2
71.52
 3.13% source/libs/executor/src/filloperator.c
2
73.6
 -0.0% source/libs/executor/src/timesliceoperator.c
2
66.02
 -0.27% source/libs/function/src/tudf.c
2
83.52
 0.0% source/libs/nodes/src/nodesEqualFuncs.c
2
63.69
 0.0% source/libs/nodes/src/nodesMatchFuncs.c
2
74.45
 -0.51% source/libs/parser/src/parInsertSml.c
2
78.31
 0.0% source/libs/planner/src/planValidator.c
2
50.75
 -0.58% source/libs/qworker/src/qwMsg.c
2
71.84
 -4.48% source/libs/stream/src/streamCheckStatus.c
2
68.09
 -1.57% source/libs/stream/src/streamHb.c
2
52.02
 -0.03% source/libs/sync/src/syncMain.c
2
72.92
 -6.25% source/libs/sync/src/syncRequestVoteReply.c
2
63.36
 0.85% source/libs/wal/src/walMeta.c
2
88.92
 -0.77% source/os/src/osString.c
2
65.28
 -1.51% source/util/src/tbloomfilter.c
2
65.73
 0.0% source/util/src/tscalablebf.c
2
78.57
 -0.95% source/util/src/ttimer.c
1
72.5
 0.06% source/common/src/msg/streamMsg.c
1
71.77
 -0.4% source/common/src/tname.c
1
63.53
 0.01% source/dnode/mgmt/mgmt_mnode/src/mmHandle.c
1
48.44
 0.0% source/dnode/mnode/impl/src/mndArbGroup.c
1
64.26
 -0.11% source/dnode/mnode/impl/src/mndDnode.c
1
72.18
 0.55% source/dnode/mnode/impl/src/mndMain.c
1
61.58
 -0.34% source/dnode/mnode/impl/src/mndProfile.c
1
79.23
 -0.4% source/dnode/mnode/impl/src/mndShow.c
1
58.31
 -0.46% source/dnode/vnode/src/tq/tqMeta.c
1
63.55
 0.06% source/dnode/vnode/src/tq/tqSink.c
1
77.34
 -0.31% source/dnode/vnode/src/tsdb/tsdbMergeTree.c
1
53.17
 -0.43% source/dnode/vnode/src/vnd/vnodeOpen.c
1
70.76
 -0.81% source/libs/executor/src/cachescanoperator.c
1
64.3
 0.24% source/libs/executor/src/countwindowoperator.c
1
60.71
 0.13% source/libs/executor/src/mergejoinoperator.c
1
56.43
 -0.91% source/libs/executor/src/timewindowoperator.c
1
65.15
 -0.11% source/libs/stream/src/streamUpdate.c
1
63.89
 2.78% source/libs/sync/src/syncAppendEntriesReply.c
1
45.71
 -0.95% source/libs/sync/src/syncIndexMgr.c
1
61.93
 -2.03% source/libs/sync/src/syncRaftCfg.c
1
51.12
 0.35% source/libs/tdb/src/db/tdbPager.c
1
70.26
 0.61% source/libs/tfs/src/tfs.c
1
77.38
 -0.34% source/os/src/osFile.c
1
81.19
 -1.98% source/os/src/osTime.c
1
83.69
 -0.39% source/util/src/tarray.c
1
96.0
 0.0% source/util/src/terror.c
Jobs
ID Job ID Ran Files Coverage
1 #3660.1 520
62.04
Source Files on build #3660