juarezr / spark-streaming-google-pubsub / 57

Committed 01 Oct 2026 07:02AM UTC coverage: 66.187% (-2.3%) from 68.457%

Build # 57

Build Type

push

github

Committed by

web-flow

Commit Message

fix: update AutoBatchTime logic for receiveTime estimation with write-first approach (#20)

- fix: update AutoBatchTime logic for receiveTime estimation with
write-first approach
- feat: add processingTime configuration to PubSubConfig to hint batch
interval on AutoBatchTime
- fix: remove unneded cap on writeBudget for AutoBatchTime

Coverage Stats

689 of 1169 branches covered (58.94%)

Branch coverage included in aggregate %.

232 of 354 new or added lines in 12 files covered. (65.54%)

15 existing lines in 1 file now uncovered.

1517 of 2164 relevant lines covered (70.1%)

0.7 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

54.55

/src/main/java/io/github/juarezr/spark/pubsub/structured/PubSubMetadataColumn.java

package io.github.juarezr.spark.pubsub.structured;

import org.apache.spark.sql.connector.catalog.MetadataColumn;
import org.apache.spark.sql.types.DataType;

final class PubSubMetadataColumn implements MetadataColumn {
  private final String name;
  private final DataType dataType;
  private final boolean nullable;

  PubSubMetadataColumn(String name, DataType dataType, boolean nullable) {
    this.name = name;
    this.dataType = dataType;
    this.nullable = nullable;
  }

  @Override
  public String toString() {
    return name + ":" + dataType + (nullable ? "?" : "");
  }

  @Override
  public String name() {
    return name;
  }

  @Override
  public DataType dataType() {
    return dataType;
  }

  @Override
  public boolean isNullable() {
    return nullable;
  }
}

1	package io.github.juarezr.spark.pubsub.structured;
2
3	import org.apache.spark.sql.connector.catalog.MetadataColumn;
4	import org.apache.spark.sql.types.DataType;
5
6	final class PubSubMetadataColumn implements MetadataColumn {
7	private final String name;
8	private final DataType dataType;
9	private final boolean nullable;
10
11	PubSubMetadataColumn(String name, DataType dataType, boolean nullable) {	1 ✔
12	this.name = name;	1 ✔
13	this.dataType = dataType;	1 ✔
14	this.nullable = nullable;	1 ✔
15	}	1 ✔
16
17	@Override
18	public String toString() {
NEW 19	return name + ":" + dataType + (nullable ? "?" : "");	×
20	}
21
22	@Override
23	public String name() {
24	return name;	1 ✔
25	}
26
27	@Override
28	public DataType dataType() {
29	return dataType;	×
30	}
31
32	@Override
33	public boolean isNullable() {
34	return nullable;	×
35	}
36	}

juarezr / spark-streaming-google-pubsub / 57

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous