Logflare / logflare

81%

main: 81%

LAST BUILD ON BRANCH feat/generational_queue_cleanup

branch: feat/generational_queue_cleanup

CHANGE BRANCH

x

Reset

• feat/generational_queue_cleanup
• abhi-improve-metadata-parsing
• abhi-serialize-instead-of-inspect
• adammokan/anl-1137-logflare-needs-to-add-support-for-fsandboxed_table_or_source
• adammokan/anl-1152-bug-alerts-route-raising-500-error
• adammokan/anl-1169-fix-clickhouse-connection-pool-wake-behavior
• adammokan/anl-1175-modify-ch-ingest-table-schema
• adammokan/anl-1175-modify-ch-ingest-table-schema-ii
• adammokan/anl-1180-more-robust-clickhouse-ingest
• adammokan/anl-1216-add-logging-and-lock-in-settings-for-clickhouse-ingestion
• adammokan/anl-1217-configure-retries-for-clickhouse-ingest-pipeline
• adammokan/anl-1219-use-tesla-client-and-tesla-retry-middleware-for-clickhouse
• adammokan/anl-1221-update-clickhouse-docs-with-ingest-flags
• adammokan/anl-1231-crash-dump-when-postgres_backend_url-contains-special
• adammokan/anl-1233-chore-follow-up-adjustments-for-tesla-client-in-clickhouse
• adammokan/anl-1234-chore-normalize-clickhouse-casing-for-consistency-in-logging
• adammokan/anl-1235-change-clickhouse-ingest-table-naming-convention
• adammokan/anl-1236-bug-fix-uuid-encoding-in-clickhouse-ingestor
• adammokan/anl-1237-chore-adjust-clickhouse-connection-timeouts-and-retries
• adammokan/anl-1243-filter-old-events-before-inserting-into-clickhouse
• adammokan/anl-1244-adjust-clickhouse-pipeline-batch-settings
• adammokan/anl-1245-tighten-up-clickhouse-logevent-iterations-when-checking
• adammokan/anl-1248-shorten-min-interval-for-alerts-from-15-5m
• adammokan/anl-1249-fix-bug-with-transform_config1-callback-in-s3adaptor
• adammokan/anl-1264-consolidate-sources-for-clickhouse-batching
• adammokan/anl-1265-filter-logevents-that-are-too-old-earlier-in-the-ingest
• adammokan/anl-1272-show-null-event_message-rows-as-no-event-message
• adammokan/anl-1282-add-consolidated_ingest-callback-to-backend-adaptor
• adammokan/anl-1283-modify-ingesteventqueue-to-support-consolidated-ingest
• adammokan/anl-1284-routing-changes-for-consolidated-ingestion
• adammokan/anl-1285-supervision-tree-for-consolidated-ingest-pipelines
• adammokan/anl-1285-supervision-tree-split
• adammokan/anl-1289-consolidated-queue-safety
• adammokan/anl-1319-modify-clickhouse-ingestion-schema
• adammokan/anl-1323-update-clickhouse-ingest-schema-to-include-source_name
• adammokan/anl-1323-update-clickhouse-ingest-schema-to-include-source_name-ii
• adammokan/anl-1327-sort-default-ingest-sources-alphabetically
• adammokan/anl-1328-add-optional-clickhouse-configuration-options-for-a-separate
• adammokan/anl-1333-additional-serialization-for-clickhouse-types
• adammokan/anl-1334-detect-log-type
• adammokan/anl-1342-transform-remap-logevents-to-otel-table-schema-shapes
• adammokan/anl-1345-pipelinequeue-per-otel-schema
• adammokan/anl-1371-configure-mapping-for-otel-logs-metrics-traces
• adammokan/anl-1372-add-support-for-mapping-to-handle-arrayslists-of
• adammokan/anl-1381-reorder-columns-in-otel_traces-table-schema
• adammokan/anl-1383-change-mapping-for-otel_logsseverity_text-in-relation-to-the
• adammokan/anl-1384-change-table-schema-for-otel_traces
• adammokan/anl-1386-adjust-clickhouse-otel-table-schemas
• adammokan/o11y-1173-chore-verify-connection-reuse-in-clickhouse-connection
• adammokan/o11y-1374-refactor-logeventlog_type-logeventevent_type
• adammokan/o11y-1378-infer-sensible-defaults
• adammokan/o11y-1380-refactor-logeventorigin_source_-attributes
• adammokan/o11y-1392-fix-encoding-issue-related-to-clickhouse-ingestion-and-port
• adammokan/o11y-1444-basic-clickhouse-native-protocol-implementation
• adammokan/o11y-1471-add-audit-logging-to-user-controller
• adammokan/o11y-1481-handle-clickhouse-error-32-properly-in-native-protocol
• adammokan/o11y-1482-spin-down-ch-connection-pool-for-native-tcp-connections
• adammokan/o11y-1486-figure-out-latency-variance-in-clickhouse-native-protocol
• adammokan/o11y-1535-improve-compression-handling-for-clickhouse-native-wire
• adammokan/o11y-1555-promote-clickhouse-simple-ingestion-schema-logic
• adammokan/o11y-1598-flatten-json-when-creating-logevent-struct
• adammokan/o11y-1607-use-logeventflattened_body-attribute-to-simplify-ch-simple
• adammokan/o11y-1634-tune-heavy-db-queries-for-flagged-ingestion-queries
• adammokan/o11y-1644-fix-grant-check-statement-for-clickhouse-backend-when-using
• adammokan/o11y-1645-add-ingested_at-column-to-clickhouse-schemas
• adammokan/o11y-1647-deeper-dig-into-clickhouse-connection-pool-handling
• adammokan/o11y-1658-clickhouse-provisioner-using-wrong-instance-if-read-only
• adammokan/o11y-1668-make-logflare-resilient-to-boolean-env-vars
• adammokan/o11y-1669-additional-histogram-buckets-for-batch-size-telemetry
• adammokan/o11y-1670-fix-typo-to-reporter_options-for-batch-size-histogram
• adammokan/o11y-1672-fix-lql-schema-field-validation-syntax-errors
• adammokan/o11y-1673-fix-lql-chart-generation-syntax-error
• adammokan/o11y-1674-support-dot-key-filtering-in-clickhouse-lql
• adammokan/o11y-1675-coerce-map-field-values-for-numeric-lql-filters
• adammokan/o11y-1787-add-clickhouse-index-to-logflare-query-templates
• adammokan/o11y-1797-use-a-60s-search-timeout-with-custom-reservations
• adammokan/o11y-1814-update-logflare-docs-access-token-link
• adammokan/o11y-1846-adjust-the-processsleep-behavior-in-the-clickhouse-native
• adammokan/o11y-1858-rework-clickhouse-pipeline-to-optimize-around-part-batches
• adammokan/o11y-1904-adjust-clickhouse-connection-pooling-for-reads
• adammokan/o11y-1934-update-logflare-docker-image-for-clickhouse
• adammokan/o11y-1935-use-async-inserts-for-stale-lane-clickhouse-batches
• adammokan/o11y-1937-cut-back-pipeline-thresholds-to-reduce-memory-footprint
• adammokan/o11y-1942-increase-clickhouse-batch-sizes
• adammokan/o11y-1945-lower-clickhouse-batch-size
• adammokan/o11y-2092-upgrade-sqlparser-rust-crate-048-054-to-support-clickhouse
• adammokan/o11y-2093-stop-sql_fmt-from-uppercasing-identifiers-logs-logs-in-sql
• adammokan/o11y-2107-re-enable-pipeline-level-retries-with-circuit-breaker-for
• adammokan/o11y-2150-add-facade_url-config-option-to-the-clickhouse-backend
• adammokan/o11y-2151-reduce-clickhouse-ingest-retries
• add-clickhouse-docs
• amokan-add-json-error-handlers
• amokan-ch-retries-follow-up
• amokan-clickhouse-query-fiesta
• amokan-clickhouse-sandbox-queries-follow-up
• amokan-clickhouse-sandboxed-queries
• amokan-fix-view-link-on-search-results
• amokan-increase-ch-conn-timeout
• amokan-lql-no-select
• amokan-lql-sandbox-queries
• amokan-sql-parser-ast-fix
• amokan-sql-parser-ast-fix-part-ii
• amokan/1-41-0-release-prep
• amokan/add-clickhouse-config-options
• amokan/add-clickhouse-wait-for-async-insert-flag
• amokan/add-test-coverage-for-ingest-event-queue-gap
• amokan/adjust-clickhouse-connection-settings
• amokan/adjust-clickhouse-retry-timeout-behavior
• amokan/back-to-72h
• amokan/better-logging-for-bq-reservation-errors
• amokan/bug-ingest-reducer-pattern-match
• amokan/ch-wait-for-async-insert
• amokan/ch_http_adjustments
• amokan/change-clickhouse-batch-size-to-50k
• amokan/change-default-bq-loc-to-eu
• amokan/clickhouse-connection-pool-adjustment
• amokan/clickhouse-native-protocol-adjustments
• amokan/clickhouse-pipeline-ack-issue
• amokan/default-ingest-bulk-update
• amokan/exclude-ch-backends-without-sources
• amokan/exclude-system-source-logging-from-metrics
• amokan/fix-ci-deps-cache
• amokan/fix-ci-issue
• amokan/fix-endpoint-query-backend-bug
• amokan/fix-search-op-suggested-fields
• amokan/follow-up-adjustments-for-3088
• amokan/increase-clickhouse-batch-size-to-100k
• amokan/lql-encoder-fix
• amokan/more-clickhouse-ingest-pipeline-logging
• amokan/more-clickhouse-logging-for-retries
• amokan/move-pipeline-settings-to-module-attrs
• amokan/otel-exporter-batch-size-config
• amokan/postgres-lql
• amokan/query-endpoint-bug
• amokan/remove-id-from-bq-cluster-keys
• amokan/reset-default-bq-loc-to-eu
• amokan/revert-otel-exporter-version
• amokan/revert-pr-3348
• amokan/schema-unlock-fix
• amokan/update-clickhouse-docker-version
• amokan/use-og-body-instead-of-flat
• avoid-atom-conversions-during-query-building
• backpressure-issue
• bb/O11Y-1892-rules-tree-fast-equals
• bb/adaptor-refactor
• bb/arrow-ipc-compression
• bb/axiom-adapter
• bb/bigquery-config
• bb/bigquery-streaming-insert-perf
• bb/bq-insert-instrumentation
• bb/bq-write-api-error-handling
• bb/cache-benchmark-and-behaviour
• bb/cainophile-cache-update
• bb/connection-testing
• bb/fix-arrow-ipc-nif-scheduling
• bb/fix-enumerable-ruletree
• bb/fix-flaky-consolidated-sup-test
• bb/ingest-rules-optimization
• bb/last9-adaptor
• bb/o11y-1314-otel-exporter-pull
• bb/o11y-1878-s3-endpoint
• bb/o11y-1890-kv-warmup-lru
• bb/o11y-1891-kv-read-replicas
• bb/o11y-2112-ingestion-for-s3-tables
• bb/o11y-707-drains-conn-testing-pt1
• bb/o11y-707-incident-io-conn-testing
• bb/o11y-707-s3-conn-testing
• bb/otlp-http-adaptor
• bb/perf-ets-info
• bb/rules-cache
• bb/via-rule-removal
• bq-json-type
• bug-schema-utils
• ch-connection-handling
• ch-ingest-batch-size-change
• ch-read-pool-init-bug
• ch_dynamic_table_naming
• chasers/log-version-metadata
• chasers/support-for-otel-kind
• chore/add-blacksmith
• chore/blacksmith-only-for-image-buidls
• chore/change-ets-metrics-to-memory-usage
• chore/ci-on-every-branch
• chore/comment-out-dialyxer-check
• chore/fix-flaky-test-attempt
• chore/flaky-test
• chore/flaky-test-sourcesup-shutdown
• chore/improve-dev-env
• chore/improve-search-lv-test
• chore/key-values-feature-flag
• chore/libcluster-pg-fixes
• chore/pr-comments-2461
• chore/reduce-rpc-flakiness
• chore/refactor-recent-logs-server
• chore/remove-admin-sources-page
• chore/remove-debugging-statement
• chore/remove-event-context-flag
• chore/remove-flow
• chore/remove-gce-strategy
• chore/remove-libcluster-pg-ssl
• chore/remove-live-monaco-editor
• chore/remove-unused-logging
• chore/remove-useless-observer-metrics
• chore/scheduler-utilization-fix
• chore/single-tenant-seeding-bug
• chore/typing-log-event-cache
• chore/update-broadway-lib
• chore/update-carg-lockfile
• chore/update-react
• chore/update_phoenix_html
• ci/make-timestamp-search-midnight-safe
• ci/use-bitnamilegacy
• claude/add-pg-sql-validation-JM8Dw
• claude/add-storage-write-api-flag-4u3C5
• claude/fix-admin-status-check-IY1DR
• claude/fix-bigquery-sql-injection-sKpqg
• claude/fix-monaco-json-security-qwh4F
• claude/fix-shutdown-auth-bypass-1VIyi
• claude/fix-stripe-webhook-security-wkE6M
• claude/fix-syslog-auth-order-TZMNE
• claude/fix-syslog-truncation-property-tests
• claude/fix-token-mass-assignment-bSXuw
• claude/fix-webhook-validation-XzPly
• claude/stabilize-flaky-tests
• clickhouse-adaptor-sql-improvements
• clickhouse-adaptor-tiny-log-removal
• clickhouse-ci-fixes
• clickhouse-connection-pools
• clickhouse-docker
• clickhouse-log-drain-phase1
• clickhouse-lql
• clickhouse-reads
• clickhouse-reads-implementation
• clickhouse-schema-changes
• clickhouse-test-setup-fix
• clickhouse_ch_client
• clickhouse_cloud_http_pool
• conductor/schemautils-preserve-metadata-string-keys
• davidwhittington/disable-bq-schema-verification
• davidwhittington/o11y-1809-metadatacleanerflatten1-allocates-intermediate-per-level
• davidwhittington/o11y-1810-logeventmake2-bigqueryschemachange-validation-via-iteraptor
• davidwhittington/o11y-1825-schemautilsto_typemap-replace-per-key-atom-conversion-with
• davidwhittington/o11y-1828-fuse-to_bigquery_column_spec-into-a-single-binary-walk
• davidwhittington/o11y-1829-bigqueryschemachangevalidate-walk-body-once-and-validate
• davidwhittington/o11y-1905-logflare-ingest_logs-droprejectedbuffer_full-counters-share
• davidwhittington/o11y-1908-replace-enummap-listflatten-with-enumflat_map-in-otel
• davidwhittington/o11y-1910-drop-dead-logeventflattened_body-field
• davidwhittington/o11y-1961-webhook-log-drains-duplicate-content-type-header-breaks-body
• davidwhittington/o11y-2019-implement-vector-protocol-as-a-grpc-ingestion-endpoint
• davidwhittington/o11y-2023-emit-git-commit-sha-in-logflare-telemetry-resource
• davidwhittington/o11y-2036-log-clickhouse-host-on-connection-and-query-errors
• davidwhittington/o11y-2097-sqlhas_restricted_sources2-crashes-on-sandboxed-endpoint
• davidwhittington/o11y-2098-ingest-crash-nestedheterogeneous-list-payloads-break
• davidwhittington/o11y-2143-alerts-api
• davidwhittington/o11y-2144-fix-ruleapischema
• davidwhittington/o11y-2144-openapi-schema-audit
• davidwhittington/o11y-2154-make-logflare-alert-backend-attachment-and-endpoint-access
• davidwhittington/o11y-2155-fix-json-error-response-media-types-in-logflare-management
• davidwhittington/schema-utils-refactor-3458
• default-backend-require-source
• default-ingest-backend-alt
• default-ingest-improvements
• default_backend_rework
• dependabot/cargo/bytes-1.11.1
• dependabot/cargo/lz4_flex-0.11.6
• dev-logflare-ui
• doc/fix-openapi
• docs-source-creation-guide
• docs/contributing-backends
• docs/db-ssl
• docs/endpoints-subquery-expansion
• docs/fix-markdown-heading
• docs/fix-typos
• docs/logpush-documentation
• docs/trace-ingestion
• docs/update-accesss-tokens-querying-docs
• docs/wait-for-render
• dynamic-tracing
• endpoint-backend-abstraction
• endpoint-backend-abstractions
• endpoint-live-ui-test-improvement
• endpoint-query-bug
• endpoint-query-ui-updates
• etienne/admin-api-updates
• feat/O11Y-1064
• feat/O11Y-635
• feat/account-level-default-ingestion
• feat/add-cluster-information-to-otel-metrics-exporter
• feat/additional-bq-projects
• feat/additional-detail-endpoints-list
• feat/adjust-chart-period
• feat/admin-api-scope
• feat/alerts-live-monaco-editor
• feat/all-metrics
• feat/allow_five_sources_for_access_token
• feat/backends-cache-refactoring
• feat/cache-clear-button
• feat/clickhouse-batch-telemetry
• feat/clickhouse-bytes-processed
• feat/copy-logs-clipboard
• feat/copy-schema-markdown
• feat/default-lql-rules
• feat/default-team-param
• feat/default-team-param-alerts
• feat/default-team-param-backends
• feat/default-team-param-billing
• feat/default-team-param-endpoints
• feat/default-team-param-key-values
• feat/default-team-param-query
• feat/deprecate-query-legacy-params
• feat/disable-tailing-source
• feat/drop-fields-transform
• feat/dynamic-bq-reservation-header
• feat/endpoint-bq-reservations
• feat/endpoint-query-versions
• feat/erlsysmon-net-kernel-logging
• feat/event-context
• feat/exclude-quick-filter
• feat/export-metrics-through-otel
• feat/format-query
• feat/gen-singleton
• feat/gensgingleton-transient
• feat/hide-null-selected-values
• feat/http-egress-span-status
• feat/humanise-timestamps
• feat/id_passing_bigquery
• feat/implement-cmdk
• feat/improve-local-timezone-viewing
• feat/increase-bq-timeout
• feat/ingest-query-scopes
• feat/ingestion-metrics-system-monitoring-refactor
• feat/interlinking
• feat/json-viewer-promoted-fields
• feat/key-values
• feat/keyvalues-cache-replication
• feat/lp-command-tz
• feat/lql-escape-quotes
• feat/lql-select-alias
• feat/metrics-pull-clean
• feat/metrics_puller
• feat/min-chart-height
• feat/modal-custom-commands
• feat/monaco_editor_for_endpoints
• feat/move-delete-source-button
• feat/new-code-editor-alerts
• feat/new-code-editor-endpoints
• feat/new-code-editor-querylive
• feat/new-code-editor-searchlive
• feat/o11y-1309
• feat/o11y-1586
• feat/oban-scheduler
• feat/open-api-improvements
• feat/open-api-spex-improvements
• feat/optional-datadog-pools
• feat/otlp-logs-ingestion
• feat/postgres-search
• feat/prompt-to-expand-range
• feat/query-clickhouse-backend
• feat/query-cost-ui-feedback
• feat/quick-filter-search
• feat/recommended-query-fields
• feat/render-event-whitespace
• feat/s3_producer_pipeline
• feat/saved-searches-modal
• feat/search-page-log-levels
• feat/search-pagination
• feat/search-results-select-fields
• feat/source-schema-modal
• feat/st-ch-ingestion
• feat/storage-write-api-improvements
• feat/storage-write-api-improvements-v2
• feat/storage-write-api-initial-v2
• feat/storage-writer-arrow-rs
• feat/store-last-team
• feat/support-ap2-datadog
• feat/team-id-param
• feat/team-roles
• feat/team-switcher
• feat/team_link_component
• feat/time-partition-bq-hour
• feat/top-processes-and-ets-tables-metrics
• feat/unrevert_metric_pull
• feat/user-system-monitoring
• fix+perf/otel-metrics-timestamp-handling
• fix-count-query
• fix-ipv6
• fix-startup
• fix/access-by-effective-user
• fix/add-create-home-team-flow
• fix/add-loki-backend-username-password
• fix/adjustments-from-last-pr
• fix/admin-accounts-list
• fix/admin-check
• fix/admin-dashboard-routing-with-home-team
• fix/admin-delete-partner
• fix/alert-schedulers-correctly-set-crontab
• fix/alerting-improvements
• fix/alerts-scheduler-termination
• fix/all-queues-buffer-check
• fix/allow-more-connections
• fix/artifact-repository-grafana
• fix/as-for-postgres-alias
• fix/avoid-creating-log-events-from-cache
• fix/backend-database-field
• fix/backends-defaults
• fix/bandit-version-bump-metrics
• fix/bigquery-adaptor-test-flake
• fix/bq-sql-extra-quotes
• fix/broken-documentation-links
• fix/broken-links
• fix/build-message-as-string
• fix/bump-dependencies
• fix/ce-role
• fix/changed-endoint-labels-clears-cache
• fix/changeset-transformation-when-field-is-present
• fix/chart-timezone
• fix/cleaner-query-lables
• fix/clickhouse-array-select-handling
• fix/clickhouse-parameters-mapping
• fix/cluster-nodename-suffix
• fix/compile-time-env-value
• fix/consistent-timestamp-precision
• fix/correctly-configure-alert-scheduler-oban
• fix/correctly-handle-no-home-team
• fix/credo-diff-against-main
• fix/dashboard-fields-bug
• fix/dashboard-metrics-poller
• fix/dashboard-navigation-team
• fix/datadog-http1
• fix/debug-modals-sql
• fix/default-backend-select
• fix/delete-source-link
• fix/deps-branch-logger
• fix/detach_metric_handlers
• fix/discord-client-empty-event-message
• fix/duplicate-fields-in-log-event-viewer
• fix/duplicate-id-warnings
• fix/duplicate_ids
• fix/endpoint-query-wrap
• fix/endpoint-run-query
• fix/endpoint-version-modal
• fix/endpoints-form-changes
• fix/endpoints-shutdown-bug
• fix/error-response
• fix/event-modal-permalink
• fix/expand-subqueries-typespec
• fix/failing-tests-styling-fixes
• fix/faster-backend-syncing
• fix/fetch-from-paritiontime-table
• fix/field-in-array
• fix/field-spacing
• fix/flaky-alerting-test
• fix/global-broadcast
• fix/goth-partitioning-sync
• fix/group-project-viewer
• fix/grpc-server-gzip
• fix/gzip-cached-query-results
• fix/handle-mutliple-union-all
• fix/handle-syn-resolve-kill
• fix/http1-for-datadog
• fix/improved-security-fixes
• fix/ingest-queue-dead-table-race
• fix/ingest-truncate-stale-table
• fix/inspect-button
• fix/left-join-metadata
• fix/legacy-artifacts-explorer
• fix/libcluster-pg
• fix/loadbalance_ets_queus
• fix/log-event-whitespace
• fix/logger-default-handler-qlen-all-envs
• fix/lql-aggregates
• fix/lql-negations
• fix/lql-preserves-timestamp-z
• fix/managed-sa-accounts-when-disabled
• fix/metadata-queries
• fix/metrics-tooltip
• fix/migrate-only-when-retired-provided
• fix/misc-bugs
• fix/missing-level-label-incoming
• fix/monaco-editor-looping-on-paste
• fix/monaco-editor-serverside-value
• fix/monaco-editor-uncontrolled
• fix/monaco-editor-web-workers
• fix/monaco-eidtor-scrolling
• fix/monaco-font-load-errors
• fix/move-ensure-source-sup-started
• fix/named-parameter-mode
• fix/navbar-session-fetching
• fix/nil-attributes
• fix/nil-suggested-fields
• fix/node-22-jquery-fix
• fix/otel-ingest-500-on-validation-rejection
• fix/otel-ingestion
• fix/otel-telemetry
• fix/partial-config-update
• fix/pii-redaction-testing
• fix/pino-logflare-link
• fix/prevent-empty-string-overwrite-source-labels
• fix/project_ref_nil_post_request
• fix/pseudo-streaming-buffer
• fix/query-only-backends-selection
• fix/race-condition-finch-pool-query
• fix/race_cond_janitor
• fix/race_with_metric_export
• fix/recent-events-touch
• fix/recommended-field-for-event-modal
• fix/reduce-conflict-logs-emitted
• fix/reduce-node-connection-attempts
• fix/refactor-log-event-live-and-component
• fix/regression-query-error
• fix/reject-ets-info-undefined-returns
• fix/rejected-events
• fix/remove-compile-env
• fix/remove-error-string
• fix/remove-grpc-logger
• fix/remove-rejected-events
• fix/remove-touch-team-user
• fix/resource-checks
• fix/restrict-tesla-env-structs-stringifying
• fix/revert-gzip-results
• fix/run-endpoint-query-with-labels
• fix/save-incidentio-backend-metadata-error
• fix/search-events-bindings
• fix/search-view-not-handling-user-context
• fix/set-verify-source
• fix/shorthand-timestamp-test-flake
• fix/shorthand-timestamp-tz
• fix/slack-signed-state
• fix/source-routing-without-sink
• fix/special-char-support
• fix/sql-postgres-translation
• fix/sql-translation-nested-cte-regexp-contains
• fix/ssl-boolean-flag-bug
• fix/ssl-ecto-repo
• fix/stale-clickhouse-config
• fix/stale-ingestion-timestamp
• fix/syn-conflict-stop-proc
• fix/syn-silencing
• fix/syn-stop-supervisor
• fix/sync-rules
• fix/table-name-format
• fix/take_pending_ids_race
• fix/team-link-in-eex-template
• fix/team-param-everywhere
• fix/team-param-handling
• fix/team-switching
• fix/test-alert-new
• fix/test-ce-role
• fix/test-query-returned-error
• fix/tooltip-escaping
• fix/typespecs
• fix/uncontrolled-endpoints-form
• fix/unnesting-metadata
• fix/update-hash-for-otel-exporter-dep
• fix/user-bq-reset-values
• fix/usersocket-logsocket-authorization-fixes
• fix/webhook-duplicate-content-type-header
• google-namespace-refactor
• hide-backend-creds-from-view
• hide-bytes-processed-when-nil
• improve-inspect-fun
• improve-utils-specs
• improving-tracing
• integracion-supabase
• investigate-sql-conversion-issue
• jordi/unredact-webhook-headers-on-update
• la/alert_component_test
• la/alerts-live-trigger-and-history-tests
• la/api-gateway-logs
• la/auth-logs
• la/auth_controller-tests
• la/backend_components_test
• la/backends-test-retry-timeout
• la/connection-pool-resolve-connections-test
• la/datetime_utils-tests
• la/delete-update_team_user_on_change_team
• la/e2e-cron-logs
• la/e2e-logs-use-supabase-js
• la/e2e-schema-modal
• la/e2e-scroll-down
• la/edge-functions-logs
• la/email-controller-coverage-test
• la/flaky-schema-modal-webkit
• la/increase-check-rate
• la/notifications-options-env-check
• la/payment_method_component
• la/payment_method_component_tests
• la/postgres-transformer-coverage
• la/postgresrest-logs
• la/postgrest-flaky-test
• la/revoke-access
• la/source_controller-tests
• la/storage-logs
• la/stripe_controller-test
• la/team_users-tests
• la/team_users_controller-test
• la/time-comparison-flaky-test
• la/utc_time_live_test
• less-ch-pipelines
• lql-filter-rule-bug
• lql-improvements
• lql-mimic-test-mock-issues
• lql-rule-serialization-bug
• lql-select-improvement
• main
• more-test-improvements
• multi-default-backends
• naive-ch-pr-additions
• ne/activate-dialyzer
• ne/admin-controller-tests
• ne/anl-1194-connection-pools-single-mode
• ne/auth-tests
• ne/backed
• ne/backend-adaptors
• ne/backend-form
• ne/backends
• ne/backends-postgres
• ne/bgrepo
• ne/bigquery-refactor
• ne/bigquery-schema-refactor
• ne/billing
• ne/billing-controller
• ne/bq-schema
• ne/caching-docker
• ne/cargo
• ne/cargo-caching
• ne/chore-socket
• ne/ci-on-main]
• ne/cluster-live
• ne/collection-terms
• ne/container-logs
• ne/content-type
• ne/coveralls
• ne/credo
• ne/credo-nesting-logflare-context_cache)
• ne/credo-nesting-logflare-logs-source_parsers-syslog_parser)
• ne/credo-nesting-logflare-logs-vercel-nimble_vercel_parser)
• ne/credo-nesting-logflare-logs-vercel-vercel_parser)
• ne/credo-nesting-logflare-lql-parser)
• ne/credo-nesting-logflare-source-text_notification_server)
• ne/credo-nesting-logflare-sql-dialect_translation)
• ne/credo-nesting-logflare-users-users)
• ne/credo-nesting-logflare_web-controllers-plugs-require_auth)
• ne/credo-nesting-logflare_web-controllers-plugs-set_team_user)
• ne/credo-nesting-logflare_web-live-vercellogdrains_live-index)
• ne/dbg
• ne/deprecated-cache-local
• ne/dialect-translation-refactor
• ne/dialyzer
• ne/dialyzer-warnings
• ne/display-helpers
• ne/docker-build
• ne/docker-test
• ne/e2e
• ne/e2e-realtime
• ne/e2e-url
• ne/ecto-atom-type
• ne/ecto-clickhouse
• ne/ecto-regex-tests
• ne/ecto-term
• ne/edit-billing-tests
• ne/elixir-1.18
• ne/elixir-ci
• ne/elixir-logger-view
• ne/enable-typing
• ne/endpoint
• ne/endpoint-controller-api
• ne/endpoints
• ne/endpoints-resolver
• ne/errors
• ne/ets
• ne/ex-coveralls
• ne/ex-machina
• ne/execute
• ne/explicit
• ne/fix-address-credo-offenses
• ne/fix-tests
• ne/fix-text-notification-server
• ne/fix-typo
• ne/flaky
• ne/flaky-backends
• ne/flaky-test
• ne/flaky-tests
• ne/flaky-timeout
• ne/gen_utils
• ne/grpc-tests
• ne/guards
• ne/ingest-event-queue
• ne/integration-tests-supabase
• ne/lib/logflare/alerting
• ne/lib/logflare/backends/adaptor/postgres_adaptor/pg_repo
• ne/lib/logflare/billing
• ne/lib/logflare/context_cache
• ne/lib/logflare/logs/log_event
• ne/lib/logflare/rules
• ne/lib/logflare/source/email_notification_server
• ne/lib/logflare_web/live/backends/backends_live
• ne/lib/logflare_web/live/source_backends_live
• ne/linting
• ne/live-rules
• ne/log-warnings
• ne/logflare-utils-parens
• ne/logflare/backends-parens
• ne/logflare/contexts-parens
• ne/logflare/google-parens
• ne/logflare/logs-parens
• ne/logflare/lql-parens
• ne/logflare/pubsub-parens
• ne/logflare/source-parens
• ne/logflare_web/controllers-parens
• ne/logflare_web/live-parens
• ne/logs-logflare-container
• ne/logs-search-live-refactor
• ne/logs-search-operations-nesting
• ne/logs-types
• ne/long-lines
• ne/max-line-length
• ne/mimic
• ne/mimic-2.3
• ne/mimic-mock-logs-bigquery
• ne/mimic-upgrade
• ne/mix-test
• ne/nested-modules
• ne/nesting
• ne/netlify
• ne/open-telemetry-sampler
• ne/open_api_spex
• ne/parens
• ne/partner-live
• ne/phoenix-live-view
• ne/phoenix-test
• ne/playwright
• ne/playwright-test
• ne/plug-set-team-if-nil
• ne/pw-screenshots
• ne/refactor-extract-system-metrics-dispatching-to-aux-function
• ne/refactor-old-templates
• ne/refactor-removed-nested-code-from-logflare-backends-adaptor
• ne/refactor-removed-nested-function-calls-in-singletenant-modul
• ne/refactor-rpc
• ne/refactor-utils
• ne/revert-missing-catch-all
• ne/rules
• ne/rules-cache
• ne/rules-followup
• ne/rust
• ne/set-plans
• ne/setup-playwright
• ne/silent_logs
• ne/sobelow
• ne/source
• ne/source-bigquery-nesting
• ne/source-controller
• ne/source-data-nesting
• ne/source-emails
• ne/source-routing-nested
• ne/source-schemas
• ne/source_controller
• ne/sources-controller
• ne/sql-sources
• ne/sql-typos
• ne/stream-data
• ne/system-metrics-parens
• ne/team-controller-tests
• ne/team-user-controller
• ne/team_user_controller
• ne/teams
• ne/tesla
• ne/test
• ne/test-parens
• ne/test-util
• ne/test-warnings
• ne/test_connection
• ne/testing-unsubscribing
• ne/tests-clickhouse
• ne/tests-coverage-context
• ne/timex
• ne/trys
• ne/tuple-warnings
• ne/types
• ne/types-gensingleton
• ne/types/alerting
• ne/types/backends
• ne/types/billing
• ne/types/endpoints
• ne/types/fixes
• ne/types/ingests
• ne/types/mismatch
• ne/types/pattern-match
• ne/types/patterns
• ne/types/unkown-types
• ne/typing
• ne/typing-unknows
• ne/typings-ex-unit
• ne/unicode
• ne/unused-files
• ne/user-controller-nesting
• ne/user-controller-tests
• ne/user-module-offenses
• ne/user-monitoring
• ne/user-prefs
• ne/users
• ne/util-tupple
• ne/vercel-tests
• ne/warning
• ne/warnings
• ne/warnings-migrations
• ne/warns
• ne/web-utils
• ne/webhooks
• non-lv-auth-checks
• observability/dynamic-pipeline-count-per-backend
• order-ch-table
• otel-improvements
• otlp-metrics
• otlp-metrics-grpc
• patch-1
• perf/add-identifiers-to-teleemtry
• perf/add-job-timeout
• perf/add-to-table-benchmark-updates
• perf/adjust_batch_size
• perf/auto-sourcesup-shutdown
• perf/backpressure-high-memory
• perf/benchmarks-ingest-event-queue
• perf/boot-init-removal
• perf/bq-ack-delete-nondefault-immediately
• perf/bq-clustering
• perf/bq-pipeline-allocation-cleanups
• perf/bq-pipeline-telemetry-handle-batch
• perf/broadcast-scaling
• perf/ch-tweaks
• perf/clickhouse-batch-retention
• perf/clickhouse-row-encoding
• perf/clickhouse_mem
• perf/client-connections
• perf/dashboard-saved-searches
• perf/endpoint-cache-tweaks
• perf/ets-table-mapper-bag
• perf/filtering-for-events
• perf/flatmap
• perf/improve-pooling
• perf/increase-batch-size
• perf/increase-chunk-size-add-to-table
• perf/increase-export-duration
• perf/increase-tail-time
• perf/ingest-queue-take-pending-ids
• perf/ingest-queue-update-counter-claim
• perf/ingesteventqueue-fix-buffer-lens-caching
• perf/ipv6
• perf/length-gt-zero
• perf/log-event-make
• perf/max-mem
• perf/memory-tweaks
• perf/nanosecond-for-recent-events-touch
• perf/only-kill-local-nodes
• perf/optional-job-creation-mode
• perf/otel-metrics-direct-export
• perf/otel-metrics-exporter-max-generations-optimizations
• perf/partition-supervisor-endpoints
• perf/partition-syn-endpoints
• perf/pool-sizes
• perf/pop_events_on_ingest
• perf/reduce-acceptors
• perf/reduce-dd-connection-counts
• perf/reduce-ingestion-broadcast
• perf/reduce-reconciliation-queries
• perf/reduce-saved-search-cache-limit
• perf/reduce-schema-module-calls
• perf/refactor-search-query-executor
• perf/relax-memory-health-check
• perf/remove-buffer-producer-telemetry
• perf/remove-get-tid-usage
• perf/remove-info-level-logging
• perf/remove-size-check-and-telemetry-queue-janitor
• perf/remove-task-nesting
• perf/round-robin-batch-size
• perf/schema-update-error
• perf/shift-syncing-to-workers
• perf/split-socket-loading
• perf/syn-gen-scope
• perf/temp-increase-erlsysmon-limits
• perf/temp-remove-job
• perf/timeouts-tweaking
• perf/truncate-non-default-backend-queues
• perf/tweak-acceptors
• perf/tweak-bandit-connection-handling
• perf/tweak-ingest-event-queue-length
• perf/tweak-memory-buffer-limit
• perf/tweak-otel-metric-exporter-opts
• perf/tweak-sourcesup-shutdown
• perf/tweak-thousand-island
• perf/tweak-user-monitoring
• perf/update_metric_exporter
• perf/user-metrics-puller-memory
• perf/user-monitoring-improvements
• perf/write-concurrency-tweaks
• pmbauer/O11Y-1914_explicit_branches_in_check_op
• pmbauer/chore_flaky_test
• poller-metrics
• pr-3667-cpu-regression-fix
• query-backend-relationship
• rd/cache-partial-broadcast
• rd/cachex-stats
• rd/format-iex-exs
• rd/gossip-docs
• rd/gossip-net_kernel-teardown
• rd/iex-exs-import_if_available
• rd/more-syslog-docs
• rd/read-from-replicas-in-caches
• rd/remove-placeholders-from-syslog-form
• rd/scheduler-observer-telemetry
• rd/syslog-credo-happy
• rd/syslog-docs-more-screenshots
• rd/syslog-no-connect-error
• rd/syslog-no-retries
• rd/syslog-telemetry
• rd/syslog-test_connection
• rd/syslog-truncate-message
• rd/syslog-worker_idle_timeout
• rd/tcp-backend
• rd/use-default-sort
• rd/webhook-clickhouse-note
• reduce-dry-run-queries
• refactor-bq-adaptor-execute-query
• refactor-ch-sup-init
• refactor-ecto-to-sql-transformations
• refactor/adaptor-query-result
• refactor/backend-query-error
• refactor/dashboard-delete-team-member
• refactor/dashboard-saved-searches
• refactor/dashboard_live
• refactor/duplicate-reservation-logging
• refactor/format-timestamp-helper
• refactor/monaco-editor-component
• refactor/move-humanize-bytes-to-logflareweb-utils
• refactor/remove-case-conditions
• refactor/remove-reset-search-event
• refactor/remove-scrivener
• refactor/remove-unused-sql-query
• refactor/remove-unused-vercel-lambda-parser
• refactor/remove_unused_code
• refactor/rename-endpoint-query
• refactor/resouce-team-id
• refactor/saved-searches-limit-uses-cache
• refactor/search-decouple-bigquery
• refactor/source-schema-from-flatmap
• refactor/user-metrics-poller
• refactor/verified-routes-warning
• refs/pull/2402/merge
• remove_ch_web_from_backend_list
• revert-3387-fix/lql-preserves-timestamp-z
• revert-3402-ruslandoga+conductor/fix-db-ssl-files
• revert/backend-controller-test-connection-reason
• revert/libcluster-postgres
• revert/metric_pull_producer
• revert/metrics_pull
• rm-empty-schema-file
• ruslandoga+conductor/db-env-docs
• ruslandoga+conductor/fix-db-ssl-files
• ruslandoga+conductor/ruslandoga-conductor/remove-dead-to-typemap
• ruslandoga+conductor/ruslandoga-conductor/remove-to-atom-typemap
• ruslandoga+conductor/ruslandoga-conductor/schema-builder-change-flag
• ruslandoga+conductor/schema-ips-reduce
• ruslandoga/ch-uuid
• s3-backend
• sentry-backend
• sql-module-refactoring
• sql-module-test-improvements
• stale-processing-timestamp-recovery
• style/formatting
• style/logger-metadata-doc
• sync-alert-jobs
• sync-alert-jobs-followup
• test/O11Y-1717
• test/clean-up-search-lv-test
• test/deterministic-assertions
• test/enable-source-test
• test/login-user-helper
• test/monaco-e2e-regression
• test/more-async-tests
• test/process-and-ets-metrics
• test/query-live-jsdom
• test/reduce_use_of_sleep
• test/remove-snapshot-ui-test
• test/remove-unused-stub
• test/saved_search_factory_lql
• test/search-ops-postgres
• test/telemetry-correctly-skips-delete-ets-tables
• test/tidy-lql-timestamp-parser
• tighter-window-for-old-events
• treq/admin-become-session-clear
• treq/fix-add-error-logging-for-ui-routes
• treq/multiple-team-users
• treq/system-sources-partition-date-fix
• treq/teamcontext-mutliple-results-error
• ui/create-admin-token
• ui/json-viewer-component
• ui/quick-filter-button
• ui/remove-dashboard-member-delete
• ui/source-description-tooltip
• ui/use-recharts
• update_clickhouse_docker_to_2512
• updated-ingest-telemetry
• wake-clickhouse-connections-after-spindown
• ziinc/add-documentation-for-cloudflare-integration
• ziinc/add-lql-operator-for-count-distinct
• ziinc/anl-1024-logflare-deprecate-all-v1-code
• ziinc/anl-1027-remove-global-broadcasting-for-buffer-rate-and-recent
• ziinc/anl-1030-disable-pii-like-ip-address-from-logs
• ziinc/anl-1049-add-dynamicpipeline-broadway-autoscaling-to-clickhouse
• ziinc/anl-1062-lf-endpoint-redact-pii-to-redact-pii-optionally
• ziinc/anl-1096-user-is-able-to-route-account-wide-bq-queries-for
• ziinc/anl-1112-logflare-reduce-schema-sampling-to-at-most-1-event-per
• ziinc/anl-1134-logflare-log-event-modal-uuid-improvements
• ziinc/anl-1177-expose-created_at-and-updated_at-in-log-drains-api
• ziinc/anl-31-logflare-sql-parser-adjust-parser-to-allow-union-all-queries
• ziinc/anl-643-logflare-add-ram-85-of-max-to-health-check
• ziinc/anl-645-logflare-sql-parser-fix-struct-parsing-error
• ziinc/anl-654-logflare-perf-tenant-upgraded-index
• ziinc/anl-658-add-coveralls-to-logflarelogflare
• ziinc/anl-661-create-cf-example-otel-tracing-worker-app-in-logflare-repo
• ziinc/anl-663-add-http_protobuf-ingestion-to-grpc-server
• ziinc/anl-674-logflare-grpc-otel-bugfixes-storage-team-ingestion
• ziinc/anl-675-logflare-grpc-otel-identity-encoding
• ziinc/anl-691-slackhookserver-fixes-customer-reporting-delayed
• ziinc/anl-697-fix-bad-cookie-definition-for-mnesia-on-logflare-instance
• ziinc/anl-708-logflare-search-automatically-change-chart-group
• ziinc/anl-712-logflare-perf-try-using-compressed-memory-option-for-local
• ziinc/anl-714-logflare-single-tenant-bugfix-stored-plan-data-may-change
• ziinc/anl-716-logflare-otel-traces-query-params-api-key-redaction
• ziinc/anl-717-logflare-access-tokens-ingestion-scopes-bug-for-grpc
• ziinc/anl-719-logflare-source-routing-bug-when-sink-is-nil
• ziinc/anl-720-logflare-performance-task-fixing-message-queue-backup-for
• ziinc/anl-721-logflare-perf-broadway-garbage-collection-for-batcher
• ziinc/anl-722-logflare-ingestion-orphaned-queues-get-cleaned-up-correctly
• ziinc/anl-723-recentlogsserver-high-memory-footprint-at-runtime
• ziinc/anl-724-logflare-account-deletion-bug-not-able-to-delete-acocunt-due
• ziinc/anl-725-logflare-alerts-quantum-job-failure-due-to-remote-node
• ziinc/anl-726-logflare-ingestion-routing-process-not-alive-for
• ziinc/anl-728-logflare-bandit-closed-errors
• ziinc/anl-730-logflare-remove-instance-external-address
• ziinc/anl-731-logflare-log-list-highlight-log-event-on-hover
• ziinc/anl-732-logflare-add-cluster-metadata-to-opentelemetry-traces
• ziinc/anl-794-logflare-alerts-incidentio-integration
• ziinc/anl-800-add-channel_name-option-to-postgres-libcluster-clustering
• ziinc/anl-818-o11y-tag-logflare-endpoint-queries-with-the-endpoint-uuid
• ziinc/anl-857-logflare-ingestion-improve-dynamicpipeline-resolving-to
• ziinc/anl-867-logflare-endpoints-does-not-register-nested-query-parameters
• ziinc/anl-879-logflare-queries-queries-get-truncated-due-to-query-params
• ziinc/anl-882-logflare-endpoints-support-large-sql-queries
• ziinc/anl-884-compress-logflare-endpoints-memory-footprint
• ziinc/anl-890-logflare-endpoints-post-support-for-body-params
• ziinc/anl-929-logflare-backends-should-not-show-secrets-in-config-for-json
• ziinc/anl-931-logflare-service-account-provisioning
• ziinc/anl-933-logflare-auth-google-auth-causing-issues-with-long-strings
• ziinc/anl-942-handle-deleted-google-accounts-bug
• ziinc/anl-944-revert-endpoints-partition-supervisor-change
• ziinc/anl-946-add-traces-for-endpoints-execution-and-endpointsresultscache
• ziinc/anl-947-add-in-metrics-for-logflare-endpoints
• ziinc/anl-948-logflare-byob-managed-service-accounts-handling
• ziinc/anl-953-endpoints-add-in-bq-billing-tagging-by-project
• ziinc/fix-flaky-test-by-adding-retry-logic
• ziinc/fix-functionclauseerror-in-slack-hook-adaptor

Build # 35d186dfaec44ff5039f745fdd2d50ef534703d0-PR-3690

Build Type

Pull #3690

github

Committed by Baishan

Commit Message

Store full event in recent so it can survive past generation cleanup. These are sweeped at 10minute intervals

Pull Request Pull Request #3690: feat: generational queue cleanup

Coverage Stats

255 of 282 new or added lines in 10 files covered. (90.43%)

14 existing lines in 5 files now uncovered.

14088 of 17406 relevant lines covered (80.94%)

9568.26 hits per line