taosdata / TDengine / #3660
67%
main: 60%

LAST BUILD BRANCH: 3.0
DEFAULT BRANCH: main
Ran
Jobs 1
Files 520
Run time 3min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 62.039% (-1.3%) from 63.314%
#3660

push

travis-ci

web-flow 
feat(stream): support stream processing for virtual tables (#30144)

* enh: add client processing

* enh: add mnode vtables processing

* enh: add mnode vtable processing

* enh: add normal child vtable support

* fix: compile issues

* fix: compile issues

* fix: create stream issues

* fix: multi stream scan issue

* fix: remove debug info

* fix: agg task and task level issues

* fix: correct task output type

* fix: split vtablescan from agg

* fix: memory leak issues

* fix: add limitations

* Update 09-error-code.md

* Update 09-error-code.md

* fix: remove usless case

* feat(stream): extract original table data in source scan task

Implemented functionality in the source task to extract data
corresponding to the virtual table from the original table using WAL.
The extracted data is then sent to the downstream merge task for further
processing.

* feat(stream): multi-way merge using loser tree in virtual merge task

Implemented multi-way merge in the merge task using a loser tree to
combine data from multiple original table into a single virtual table.
The merged virtual table data is then pushed downstream for further
processing.  Introduced memory limit handling during the merge process
with configurable behavior when the memory limit is reached.

* fix(test): remove useless cases

---------

Co-authored-by: dapan1121 <wpan@taosdata.com>
Co-authored-by: Pan Wei <72057773+dapan1121@users.noreply.github.com>

154078 of 317582 branches covered (48.52%)

Branch coverage included in aggregate %.

313 of 2391 new or added lines in 34 files covered. (13.09%)

26134 existing lines in 205 files now uncovered.

240261 of 318051 relevant lines covered (75.54%)

16655189.27 hits per line

New Missed Lines in Diff

Lines Coverage ∆ File
1
69.67
 0.33% source/util/src/tlrucache.c
2
61.14
 0.18% source/dnode/mnode/impl/src/mndStream.c
2
59.77
 -1.29% source/dnode/vnode/src/vnd/vnodeSvr.c
2
72.71
 -0.29% source/libs/parser/src/parUtil.c
3
80.87
 -0.34% source/libs/planner/src/planLogicCreater.c
4
54.37
 -3.13% source/libs/executor/src/operator.c
9
64.85
 -1.06% source/libs/stream/src/streamCheckpoint.c
10
80.31
 0.39% source/libs/planner/src/planSpliter.c
11
75.03
 -0.99% source/libs/parser/src/parTranslater.c
12
47.3
 1.85% source/libs/catalog/src/ctgCache.c
20
81.86
 -0.53% source/libs/nodes/src/nodesUtilFuncs.c
24
49.37
 3.73% source/libs/catalog/src/ctgRemote.c
25
50.5
 2.29% source/libs/qcom/src/querymsg.c
28
71.84
 -4.48% source/libs/stream/src/streamCheckStatus.c
31
63.73
 -1.49% source/libs/executor/src/scanoperator.c
50
55.33
 -1.01% source/libs/catalog/src/ctgUtil.c
73
62.82
 -7.41% source/libs/stream/src/streamDispatch.c
76
48.62
 -1.14% source/libs/catalog/src/ctgAsync.c
103
49.0
 -8.94% source/dnode/vnode/src/vnd/vnodeQuery.c
115
60.05
 -8.28% source/libs/stream/src/streamTask.c
121
68.55
 -1.47% source/common/src/msg/tmsg.c
156
66.6
 -5.02% source/libs/nodes/src/nodesCodeFuncs.c
352
48.47
 -16.41% source/dnode/vnode/src/tq/tqRead.c
416
0.0
 source/libs/executor/src/virtualtablescanoperator.c
432
44.38
 -32.22% source/dnode/mnode/impl/src/mndScheduler.c

Uncovered Existing Lines

Lines Coverage ∆ File
1
72.5
 0.06% source/common/src/msg/streamMsg.c
1
71.77
 -0.4% source/common/src/tname.c
1
63.53
 0.01% source/dnode/mgmt/mgmt_mnode/src/mmHandle.c
1
48.44
 0.0% source/dnode/mnode/impl/src/mndArbGroup.c
1
64.26
 -0.11% source/dnode/mnode/impl/src/mndDnode.c
1
72.18
 0.55% source/dnode/mnode/impl/src/mndMain.c
1
61.58
 -0.34% source/dnode/mnode/impl/src/mndProfile.c
1
79.23
 -0.4% source/dnode/mnode/impl/src/mndShow.c
1
58.31
 -0.46% source/dnode/vnode/src/tq/tqMeta.c
1
63.55
 0.06% source/dnode/vnode/src/tq/tqSink.c
1
77.34
 -0.31% source/dnode/vnode/src/tsdb/tsdbMergeTree.c
1
53.17
 -0.43% source/dnode/vnode/src/vnd/vnodeOpen.c
1
70.76
 -0.81% source/libs/executor/src/cachescanoperator.c
1
64.3
 0.24% source/libs/executor/src/countwindowoperator.c
1
60.71
 0.13% source/libs/executor/src/mergejoinoperator.c
1
56.43
 -0.91% source/libs/executor/src/timewindowoperator.c
1
65.15
 -0.11% source/libs/stream/src/streamUpdate.c
1
63.89
 2.78% source/libs/sync/src/syncAppendEntriesReply.c
1
45.71
 -0.95% source/libs/sync/src/syncIndexMgr.c
1
61.93
 -2.03% source/libs/sync/src/syncRaftCfg.c
1
51.12
 0.35% source/libs/tdb/src/db/tdbPager.c
1
70.26
 0.61% source/libs/tfs/src/tfs.c
1
77.38
 -0.34% source/os/src/osFile.c
1
81.19
 -1.98% source/os/src/osTime.c
1
83.69
 -0.39% source/util/src/tarray.c
1
96.0
 0.0% source/util/src/terror.c
2
87.88
 -2.44% include/common/tcommon.h
2
63.02
 -1.04% source/client/src/clientSmlJson.c
2
56.95
 -0.61% source/dnode/mnode/impl/src/mndStreamTransAct.c
2
69.38
 -0.19% source/dnode/mnode/impl/src/mndSubscribe.c
2
85.43
 -0.36% source/dnode/mnode/sdb/src/sdbHash.c
2
78.97
 -0.26% source/dnode/vnode/src/tsdb/tsdbDataFileRW.c
2
70.86
 0.41% source/dnode/vnode/src/tsdb/tsdbReadUtil.c
2
71.52
 3.13% source/libs/executor/src/filloperator.c
2
73.6
 -0.0% source/libs/executor/src/timesliceoperator.c
2
66.02
 -0.27% source/libs/function/src/tudf.c
2
83.52
 0.0% source/libs/nodes/src/nodesEqualFuncs.c
2
63.69
 0.0% source/libs/nodes/src/nodesMatchFuncs.c
2
74.45
 -0.51% source/libs/parser/src/parInsertSml.c
2
78.31
 0.0% source/libs/planner/src/planValidator.c
2
50.75
 -0.58% source/libs/qworker/src/qwMsg.c
2
71.84
 -4.48% source/libs/stream/src/streamCheckStatus.c
2
68.09
 -1.57% source/libs/stream/src/streamHb.c
2
52.02
 -0.03% source/libs/sync/src/syncMain.c
2
72.92
 -6.25% source/libs/sync/src/syncRequestVoteReply.c
2
63.36
 0.85% source/libs/wal/src/walMeta.c
2
88.92
 -0.77% source/os/src/osString.c
2
65.28
 -1.51% source/util/src/tbloomfilter.c
2
65.73
 0.0% source/util/src/tscalablebf.c
2
78.57
 -0.95% source/util/src/ttimer.c
3
61.64
 -0.59% source/dnode/mgmt/mgmt_vnode/src/vmInt.c
3
51.92
 -0.13% source/dnode/vnode/src/tsdb/tsdbFS2.c
3
65.78
 -0.75% source/dnode/vnode/src/vnd/vnodeSync.c
3
49.37
 3.73% source/libs/catalog/src/ctgRemote.c
3
78.59
 -1.22% source/libs/executor/src/eventwindowoperator.c
3
58.74
 0.0% source/libs/qworker/src/qwUtil.c
3
39.66
 0.17% source/libs/scheduler/src/schJob.c
3
64.65
 0.5% source/libs/transport/src/transComm.c
3
84.77
 -0.57% source/os/src/osThread.c
3
66.67
 0.02% tools/taos-tools/src/benchInsert.c
4
51.76
 -0.49% source/client/src/clientMonitor.c
4
79.69
 -0.27% source/common/src/ttime.c
4
70.43
 -1.02% source/dnode/vnode/src/tsdb/tsdbIter.c
4
55.81
 0.04% source/libs/transport/src/transSvr.c
4
60.28
 -0.35% source/libs/wal/src/walRead.c
5
59.37
 0.02% source/client/src/clientTmq.c
5
55.26
 -0.62% source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
5
63.63
 -1.63% source/dnode/vnode/src/tsdb/tsdbFSet2.c
5
55.3
 0.31% source/libs/executor/src/dataDispatcher.c
5
86.32
 1.1% source/libs/nodes/src/nodesTraverseFuncs.c
5
59.05
 -5.71% source/libs/qworker/src/qwMem.c
5
64.27
 -0.13% source/libs/tdb/src/db/tdbBtree.c
5
90.95
 -2.83% source/util/src/thashutil.c
6
62.4
 0.99% source/client/src/clientHb.c
6
71.79
 -25.64% source/dnode/mnode/impl/src/mndIndexCom.c
7
56.28
 -0.02% source/libs/transport/src/transCli.c
7
80.24
 -4.03% source/util/src/tdigest.c
8
59.13
 -0.96% source/libs/scheduler/src/schTask.c
10
77.11
 -0.1% source/libs/executor/src/querytask.c
10
61.05
 -2.26% source/libs/wal/src/walWrite.c
11
66.15
 0.17% source/libs/executor/src/tsort.c
12
58.86
 -1.33% source/dnode/vnode/src/sma/smaRollup.c
12
82.93
 0.36% source/libs/parser/src/parAuthenticator.c
13
73.03
 13.49% source/common/src/ttypes.c
13
55.32
 -0.08% source/dnode/mnode/impl/src/mndPerfSchema.c
13
37.8
 -1.46% source/libs/parser/src/parInsertStmt.c
14
56.71
 -0.44% source/dnode/mnode/impl/src/mndInfoSchema.c
14
71.7
 -1.43% source/dnode/mnode/impl/src/mndStreamUtil.c
14
73.67
 1.88% source/dnode/vnode/src/tq/tqStreamTask.c
15
53.66
 -7.32% source/dnode/vnode/src/sma/smaEnv.c
16
59.11
 -0.37% source/dnode/mnode/impl/src/mndVgroup.c
16
62.98
 -2.05% source/util/src/tcompression.c
17
65.23
 -4.54% source/libs/stream/src/streamSliceState.c
18
72.93
 -1.07% source/client/src/clientSml.c
18
80.23
 0.42% source/libs/planner/src/planScaleOut.c
19
62.45
 -1.08% source/dnode/mnode/impl/src/mndTrans.c
19
84.38
 -2.96% utils/tsim/src/simExec.c
20
80.0
 3.33% source/libs/function/src/detail/tavgfunction.c
21
71.37
 0.78% source/dnode/mgmt/node_mgmt/src/dmTransport.c
22
85.69
 1.28% source/common/src/tcol.c
23
57.69
 -6.46% source/libs/parser/src/parser.c
26
56.61
 1.6% source/libs/qcom/src/queryUtil.c
29
81.08
 2.53% source/libs/parser/src/parTokenizer.c
29
62.45
 -10.15% source/libs/planner/src/planner.c
32
63.58
 -0.63% source/dnode/mnode/impl/src/mndIndex.c
33
65.55
 -1.51% source/libs/executor/src/executorInt.c
33
68.66
 -0.82% source/libs/executor/src/projectoperator.c
35
74.6
 0.56% source/dnode/vnode/src/tq/tqUtil.c
37
74.47
 -0.2% source/libs/executor/src/tfill.c
38
68.93
 -0.53% source/libs/parser/src/parInsertUtil.c
41
68.91
 0.25% source/libs/executor/src/groupoperator.c
42
63.95
 -9.56% source/libs/stream/src/streamData.c
44
18.93
 -0.06% source/dnode/mnode/impl/src/mndDump.c
44
77.59
 -0.0% source/libs/stream/src/streamQueue.c
45
69.16
 0.06% source/dnode/vnode/src/tsdb/tsdbCacheRead.c
45
75.34
 0.61% source/libs/executor/src/mergeoperator.c
46
77.24
 -1.77% source/libs/function/src/functionMgt.c
52
57.65
 0.02% source/libs/executor/src/groupcacheoperator.c
52
61.36
 -1.1% source/libs/function/src/tpercentile.c
55
64.9
 -4.04% source/libs/executor/src/streameventwindowoperator.c
56
74.48
 -0.47% source/libs/executor/src/streamfilloperator.c
59
62.07
 2.31% source/libs/stream/src/streamStartHistory.c
60
39.83
 -8.69% include/common/tmsg.h
60
63.96
 0.86% source/libs/executor/src/streamintervalsliceoperator.c
61
0.0
 -66.67% source/dnode/vnode/src/meta/metaSma.c
64
82.78
 -1.31% source/libs/parser/src/parAstParser.c
68
67.22
 -0.2% source/dnode/vnode/src/tsdb/tsdbUtil.c
68
62.26
 -11.11% source/libs/stream/src/streamSched.c
73
48.47
 -16.41% source/dnode/vnode/src/tq/tqRead.c
74
65.86
 0.46% source/libs/function/src/detail/tminmax.c
77
77.2
 0.01% source/libs/planner/src/planUtil.c
79
64.5
 -5.64% source/client/src/clientStmt.c
81
44.38
 -32.22% source/dnode/mnode/impl/src/mndScheduler.c
81
50.5
 2.29% source/libs/qcom/src/querymsg.c
87
34.21
 0.29% source/client/src/clientStmt2.c
89
62.23
 -1.95% source/dnode/vnode/src/tq/tq.c
89
69.71
 -4.09% source/libs/stream/src/streamState.c
91
76.81
 3.14% tools/shell/src/shellEngine.c
92
60.63
 0.05% source/libs/qworker/src/qworker.c
103
68.08
 -1.02% source/libs/executor/src/executil.c
105
54.37
 -3.13% source/libs/executor/src/operator.c
105
79.97
 0.6% source/libs/function/src/builtins.c
111
64.85
 -1.06% source/libs/stream/src/streamCheckpoint.c
112
73.43
 -1.1% source/libs/nodes/src/nodesCloneFuncs.c
113
56.84
 -11.51% source/dnode/vnode/src/meta/metaEntry.c
124
64.96
 -0.04% source/libs/executor/src/executor.c
141
59.96
 0.01% source/client/src/clientRawBlockWrite.c
142
0.0
 -54.64% source/dnode/vnode/src/sma/smaTimeRange.c
146
72.71
 -0.29% source/libs/parser/src/parUtil.c
151
80.31
 0.39% source/libs/planner/src/planSpliter.c
152
66.62
 2.96% source/libs/command/src/command.c
153
67.23
 -0.32% source/dnode/mnode/impl/src/mndDb.c
153
74.56
 -1.52% source/libs/executor/src/streamtimesliceoperator.c
166
59.63
 -6.89% source/dnode/vnode/src/meta/metaTable.c
166
59.95
 0.51% source/libs/stream/src/streamMeta.c
166
60.05
 -8.28% source/libs/stream/src/streamTask.c
176
62.42
 -10.84% source/libs/stream/src/streamExec.c
182
19.01
 3.41% source/libs/executor/src/streamexecutorInt.c
183
62.25
 -0.37% source/dnode/vnode/src/tqCommon/tqCommon.c
183
49.0
 -8.94% source/dnode/vnode/src/vnd/vnodeQuery.c
183
80.87
 -0.34% source/libs/planner/src/planLogicCreater.c
190
56.13
 -0.55% source/dnode/vnode/src/meta/metaQuery.c
193
59.77
 -1.29% source/dnode/vnode/src/vnd/vnodeSvr.c
194
48.89
 -1.57% source/libs/scalar/src/sclvector.c
197
27.92
 8.9% source/libs/catalog/src/catalog.c
199
62.82
 -7.41% source/libs/stream/src/streamDispatch.c
210
81.86
 -0.53% source/libs/nodes/src/nodesUtilFuncs.c
212
67.34
 0.04% source/common/src/tglobal.c
221
67.84
 -0.63% source/libs/scalar/src/scalar.c
226
54.16
 -6.0% source/libs/executor/src/exchangeoperator.c
238
55.33
 -1.01% source/libs/catalog/src/ctgUtil.c
267
56.85
 -1.3% source/libs/command/src/explain.c
268
73.99
 -0.45% source/libs/parser/src/parInsertSql.c
283
52.08
 -0.06% source/libs/stream/src/streamBackendRocksdb.c
286
57.07
 -1.09% source/client/src/clientMain.c
295
82.74
 -0.36% source/libs/nodes/src/nodesMsgFuncs.c
306
60.62
 -3.24% source/libs/stream/src/tstreamFileState.c
306
55.21
 -0.8% source/util/src/tcompare.c
314
54.79
 -13.77% source/libs/stream/src/streamSessionState.c
338
64.14
 -0.47% source/client/src/clientImpl.c
357
53.52
 -2.78% source/libs/scalar/src/sclfunc.c
378
69.76
 0.02% source/dnode/vnode/src/tsdb/tsdbRead2.c
399
68.05
 0.01% source/libs/parser/src/parAstCreater.c
399
69.04
 -0.75% source/libs/planner/src/planPhysiCreater.c
410
64.51
 -1.09% source/common/src/tdatablock.c
418
61.14
 0.18% source/dnode/mnode/impl/src/mndStream.c
421
61.0
 0.16% source/common/src/tdataformat.c
435
48.41
 -12.11% source/dnode/mnode/impl/src/mndSma.c
445
47.46
 -13.96% source/dnode/vnode/src/meta/metaEntry2.c
451
48.62
 -1.14% source/libs/catalog/src/ctgAsync.c
469
61.63
 1.02% source/dnode/vnode/src/tsdb/tsdbCache.c
503
36.09
 -18.55% source/libs/executor/src/dynqueryctrloperator.c
521
67.11
 0.4% source/dnode/mnode/impl/src/mndStb.c
531
47.3
 1.85% source/libs/catalog/src/ctgCache.c
533
71.35
 0.13% source/libs/executor/src/streamtimewindowoperator.c
537
43.73
 -10.99% source/dnode/vnode/src/meta/metaTable2.c
591
74.35
 -1.42% source/libs/planner/src/planOptimizer.c
629
55.39
 -4.64% source/libs/executor/src/sysscanoperator.c
654
71.67
 -0.72% source/libs/function/src/builtinsimpl.c
698
66.18
 -0.61% source/libs/scalar/src/filter.c
838
63.73
 -1.49% source/libs/executor/src/scanoperator.c
929
62.52
 0.21% tools/taos-tools/src/taosdump.c
975
66.6
 -5.02% source/libs/nodes/src/nodesCodeFuncs.c
1022
68.55
 -1.47% source/common/src/msg/tmsg.c
1907
75.03
 -0.99% source/libs/parser/src/parTranslater.c
Jobs
ID Job ID Ran Files Coverage
1 #3660.1 520
62.04
Source Files on build #3660