24.12.1

Various fixes & improvements

• Revert "fix(eap-spans): Add an index on project_id (#6695)" (9f5c3f5) by @getsentry-bot
• fix(eap-spans): Add an index on project_id (#6695) by @phacops
• 984: scrub eap_spans_str_attrs (#6694) by @kylemumma
• fix(inc984): scrub the correct bucket for sentry.user (#6693) by @xurui-c
• fix(inc984): scrub the correct bucket for sentry.user.ip (#6692) by @volokluev
• inc984: scrub user from eap_spans (#6691) by @xurui-c
• inc984: scrub user from spans (#6689) by @xurui-c
• fix(rust): Honor exit code (#6674) by @untitaker
• ref(admin): lw delete related system queries (#6685) by @MeredithAnya
• chore: Bump rust-toolchain (#6688) by @untitaker
• chore(devservices): Bumping the version of devservices to latest (#6682) by @IanWoodard
• fix(devservices): Add orchestrator devservices label to clickhouse (#6687) by @hubertdeng123
• fix(eap): Fix divide by 0 errors caused when the sample count is 0 (#6681) by @davidtsuk
• meta: Bump new development version (4465733)
• ref(lw-deletes): add project_id killswitch and some logging (#6677) by @MeredithAnya
• docs: update CH supported versions (#6683) by @MeredithAnya
• feat(inc-984): Cleanup EAP spans (#6676) by @xurui-c
• fix(admin): Allow special characters in SYSTEM/OPTIMIZE queries (#6680) by @evanh
• feat(eap): Use weighted average instead of simple average for calculating average sampling rate (#6678) by @davidtsuk
• ref(lw-deletes): concurrent allocation policy requires org id (#6679) by @MeredithAnya

24.12.0

Various fixes & improvements

• feat(inc-984): store project ids list in dictionary in scrub job (#6675) by @volokluev
• ref(lw-deletes): enforce ratelimiter (#6644) by @MeredithAnya
• fix(admin): Allow KILL MUTATION commands in sudo mode (#6672) by @evanh
• fix(inc984): align start/end timestamp to partition boundaries (#6670) by @volokluev
• chore(deps): bump relay from 0.9.2 to 0.9.4 (#6660) by @jjbayer
• feat(inc984): make mutation condition simpler (#6669) by @volokluev
• chore: Bump Arroyo to 2.19.5 (#6666) by @ayirr7
• ref: bump sentry-arroyo to 2.19.4 (#6663) by @getsentry-bot
• fix(eap-alerts): Fix subscriptions referrer for eap alerts (#6662) by @shruthilayaj
• chore(api): Do not log healthcheck error if downfile exists (#6635) by @untitaker
• feat(eap): add additional validation for group by (#6659) by @davidtsuk
• feat(eap): add default value to virtual column (#6657) by @davidtsuk
• ref: bump sentry-arroyo to 2.19.3 (#6656) by @getsentry-bot
• Implement filter offset for attribute keys API (#6618) by @xurui-c
• feat: make sentry RPC instrumentation more specific to the endpoint (#6654) by @kylemumma
• fix(consumers): Respect 60 day retention days period (#6631) by @volokluev
• feat: add missing example in admin rpc tool (#6647) by @kylemumma
• hotfix(inc-984): Add manual job to scrub IPs from spans (#6649) by @volokluev
• feat: support 15 minute granularity on eap time series RPC (#6645) by @kylemumma
• fix(eap): Fix divide by 0 bug (#6653) by @davidtsuk
• fix: run sentry tests when RPC changes (#6652) by @colin-sentry
• meta: Bump new development version (60ff544)
• chore(eap-spans): Take advantage of parallel reads (#6579) by @phacops

24.11.2

Various fixes & improvements

• feat(eap): Bump max timeseries buckets (#6630) by @volokluev
• fix(scripts): Automatically create local tables before MV (#6648) by @untitaker
• feat(eap): Implement confidence intervals for percentiles (#6634) by @davidtsuk
• fix: admin rpc endpoint selector bug fix (#6641) by @kylemumma
• ref(admin): Allow merge function in FROM clause (#6643) by @evanh
• fix: Actual Rust support in bump-version (#6640) by @untitaker
• fix 0006_sorting_key_change migration (#6633) by @MeredithAnya
• chore: Bump Arroyo 2.19.2 (#6639) by @ayirr7
• chore: Bump Arroyo to 2.19.1 (#6632) by @ayirr7
• chore(devservices): Bumping the version of devservices to latest (#6629) by @IanWoodard
• feat(eap-api): support in conditions in TraceItemFilter (#6623) by @kylemumma
• Revert "ref(ci): Disable self-hosted e2e CI due to resource constraints" (#6622) by @hubertdeng123
• feat(devservices): Use https for repo links (#6627) by @hubertdeng123
• chore: Bump rust-arroyo to published crate (#6614) by @untitaker
• feat(label): adding label to container (#6628) by @IanWoodard
• consumer: Add flag to configure DLQ buffer limit (#6626) by @ayirr7
• feat(runtime_config): A way to add static descriptions to config items (#6624) by @untitaker
• fix(admin): add example RPC call for the endpoints missing it (#6615) by @kylemumma
• feat(eap-api): Use signs in count, average, and sum calculations (#6613) by @davidtsuk
• feat(devservices): Add healthchecks to clickhouse and snuba (#6616) by @hubertdeng123
• feat(memory): Add script to debug consumer OOM issues (#6585) by @nikhars
• ref(ci): Disable self-hosted e2e CI due to resource constraints (#6620) by @hubertdeng123
• ref(admin): Allow clusterAllReplicas in system queries (#6619) by @evanh
• ref(profiling): start the profile_chunks consumer when ENABLE_PROFILES_CONSUMER is true (#6607) by @viglia

Plus 11 more

24.11.1

Various fixes & improvements

• ref(clickhouse-24.3): allow_suspicious_primary_key for eap migration (#6602) by @MeredithAnya
• chore(devservices): Bumping the version of devservices to latest (#6596) by @IanWoodard
• feat(consumers): Quantized rebalancing in Rust (#6595) by @untitaker
• chore(metrics-summaries): Remove metrics summaries consumer code (#6590) by @phacops
• feat(eap): Extract profile id for eap spans (#6597) by @Zylphrex
• fix(codeowners): Code owners for eap processor (#6598) by @Zylphrex
• fix(eap): fix bug where we pass non str group by mapping to timeseries (#6593) by @davidtsuk
• fix(eap-api): delimeter for alias (#6592) by @xurui-c
• feat(eap): Add confidence interval calculations for count (#6568) by @davidtsuk
• RPC handles same name attributes (#6581) by @xurui-c
• log(optimize): after optimize, log how long it took w/partition (#6591) by @onewland
• feat(devservices): Add restart settings and use external devservices network (#6587) by @hubertdeng123
• chore(deps): bump actions/checkout from 2 to 4 (#6554) by @onkar
• chore(devservices): Bumping the version of devservices to latest (#6582) by @IanWoodard
• chore(ci): Upgrade ClickHouse to v23.8 for CI (#6584) by @phacops
• feat(subscriptions): Add a create subscription rpc endpoint (#6571) by @shruthilayaj
• fix(script): Wait for futures when producing to kafka (#6583) by @nikhars
• fix: convert max_batch_time seconds (#6580) by @MeredithAnya
• fix: dont set group.instance.id incorrectly (#6578) by @MeredithAnya
• chore(eap-spans): Increase the number of replicas to scan in parallel (#6577) by @phacops
• V1 RPC of TraceItemAttributeValues (#6563) by @xurui-c
• meta: Bump new development version (9b92334)
• build(deps): bump types-python-dateutil from 2.8.19 to 2.8.19.14 (#6557) by @onkar
• build(deps-dev): bump webpack-cli from 4.10.0 to 5.1.4 in /snuba/admin (#6558) by @onkar

Plus 4 more

24.11.0

Various fixes & improvements

• feat(consumers): rust consumers quantized rebalance (#6561) by @volokluev
• chore(deps): bump docker/setup-buildx-action from 2 to 3 (#6553) by @onkar
• ref(deletes): add lw-deletions-search-issues-consumer to deploy.sh (#6567) by @MeredithAnya
• fix(admin): ignore gather_profile_events for certain storages (#6566) by @MeredithAnya
• chore(deps): bump getsentry/action-github-app-token from 2.0.0 to 3.0.0 (#6549) by @onkar
• ref(deletes): add use_bulk_deletes runtime config (#6560) by @MeredithAnya
• feat(devservices): Only expose ports to localhost (#6565) by @hubertdeng123
• chore(deps): bump types-pyyaml from 6.0.11 to 6.0.12.20240808 (#6552) by @onkar
• chore(deps): bump getsentry/action-migrations from 1.0.8 to 1.2.2 (#6556) by @onkar
• feat(eap): Add support for extrapolation (#6536) by @davidtsuk
• feat(devservices): Bump devservices version (#6507) by @hubertdeng123
• chore(eap-spans): Remove unused tables (#6544) by @phacops
• chore(eap-spans): Switch test to use new spans table (#6550) by @phacops
• [CapMan visibility] rejected queries are ran with 0 threads (#6545) by @xurui-c
• fix: ENABLE_LW_DELETIONS_CONSUMER default to False (#6546) by @MeredithAnya
• ref(deletes): bulk delete consumer (#6510) by @MeredithAnya
• chore(deps): bump packaging from 21.3 to 24.1 (#6540) by @onkar
• chore(deps): bump progressbar2 from 4.0.0 to 4.2.0 (#6534) by @onkar
• chore(deps): bump jsonschema from 4.16.0 to 4.23.0 (#6539) by @onkar
• chore(deps): bump google-github-actions/auth from 1 to 2 (#6541) by @onkar
• build(deps): bump sentry-usage-accountant from 0.0.10 to 0.0.11 (#6542) by @onkar
• build(deps): bump actions/setup-node from 3 to 4 (#6543) by @onkar
• Fix sampling weight calculation (#6537) by @davidtsuk
• chore(deps): bump types-setuptools from 65.3.0 to 74.1.0.20240907 (#6526) by @onkar

Plus 102 more

24.10.0

Various fixes & improvements

• ref(functions): refactor functions processor and storage (#6411) by @viglia
• fix(eap): remove flask stuff from rpc (#6410) by @volokluev
• fix(admin): Handle NaN in admin results (#6412) by @evanh
• fix(database-clusters): Use a lock when creating connection (#6407) by @davidtsuk
• feat(rpc): handle exceptions by returning a proto and non-200 (#6402) by @colin-sentry
• ref(eap): Clean up the mutations interface (#6344) by @untitaker
• feat(admin): Show data in clickhouse tracing tool (#6406) by @untitaker
• feat(database-clusters): Use a ThreadPoolExecutor to speed up node fetching (#6405) by @davidtsuk
• feat(profiling): add new columns to the raw_functions table (#6398) by @viglia
• fix(database-clusters): Remove Query Node column (#6404) by @davidtsuk
• feat(eap): Port TraceItemTable endpoint to v1 (#6401) by @volokluev
• feat: Add EAP span subscriptions for devserver (#6399) by @shruthilayaj
• ref(deletes): increase max rows to 100000 (#6400) by @MeredithAnya
• Revert "feat: Add EAP span subscriptions for devserver (#6396)" (ff04932) by @getsentry-bot
• feat: Add EAP span subscriptions for devserver (#6396) by @shruthilayaj
• feat(eap): record rpc endpoint success and timing (#6392) by @volokluev
• feat(job-runner) view logs in custom jobs admin page (#6394) by @onewland
• feat(job-runner): persist logs in Redis for easier viewing in snuba admin (#6387) by @onewland
• fix(devservices): Fixing devservices config (#6393) by @IanWoodard
• fix(database-clusters): Use host name instead of address when fetching system settings (#6391) by @davidtsuk
• fix: better error message in admin system query (#6390) by @kylemumma
• feat(database-clusters): Add server settings for each node and additional data (#6386) by @davidtsuk
• feat(job-runner): run job from snuba admin (#6385) by @onewland
• feat(capman): remove legacy cap on threads (#6389) by @volokluev

Plus 73 more

24.9.0

Various fixes & improvements

• Update migrations list command to show migrations that no longer exist in the codebase (#6299) by @davidtsuk
• metric(consumer): Add a metric to track the size of individual spans (#6300) by @ayirr7
• feat(rpc): Update tags list rpc (#6301) by @Zylphrex
• feat(eap): add virtual column support (#6292) by @volokluev
• tweak(eap): Allow more memory usage for eap spans (#6298) by @volokluev
• ref(doc): add documentation for the ReadinessState enum (#6295) by @viglia
• feat(eap): Start ingesting data into sample_weight_2 column (#6290) by @colin-sentry
• Update docker entrypoint to run heaptrack (#6273) by @ayirr7
• fix(eap): Switch to sampling_weight_2 in entity (#6287) by @colin-sentry
• bug(query): Run entity validators in composite query pipeline (#6285) by @enochtangg
• feat(eap): make mapContains work with EAP dataset (#6284) by @colin-sentry
• feat(job-runner): create a new snuba jobs command (#6281) by @xurui-c
• feat(eap): Shard meta tables by trace ID (#6286) by @colin-sentry
• fix(eap): Make span_id be returned as a string correctly (#6283) by @colin-sentry
• feat(job-runner): scaffolding for job manifest testing (#6282) by @onewland
• bug(admin): Fix invalid query error alerting in snuba admin (#6280) by @enochtangg
• Fixing Snuba Admin trace UI error. (#6278) by @nachivrn
• feat(eap): Add a processor that allows you to do mapKeys on attr_str (#6277) by @colin-sentry
• cleanup(capman): remove legacy table rate limits (#6274) by @volokluev
• Fixing Snuba Admin trace UI error. (#6276) by @nachivrn
• hackweek(snuba-admin): MQL query tool (#6235) by @enochtangg
• feat(eap): Endpoint to get the tags available for a project (#6270) by @colin-sentry
• feat(sudo): issue slack notifications when sudo mode is used (#6271) by @volokluev
• chore(eap): Add entities and storages for EAP span meta tables (#6269) by @colin-sentry

Plus 60 more

24.8.0

Various fixes & improvements

• switch readiness state to limited to skip migrations and unblock CI (#6210) by @volokluev
• fix: Add DropIndices back as it was removed after a revert (#6209) by @phacops
• fix(eap): fix number migration collision (#6208) by @volokluev
• Revert "fix(eap-spans): Drop ineffective indices to speed up insertion (#6206)" (6882653) by @getsentry-bot
• ref(snuba-deletes) don't run delete queries when 0 rows need to be deleted (#6199) by @xurui-c
• fix(eap-spans): Drop ineffective indices to speed up insertion (#6206) by @phacops
• ref(eap): Remove materialized view (#6205) by @evanh
• fix: deletes api, change success response format, fix admin bug (#6197) by @kylemumma
• feat(eap): stub API for aggregate bucket request (#6204) by @volokluev
• ref(eap): Change sampling_weight to a UInt (#6190) by @evanh
• feat(eap): Aggregate request protobuf endpoint (#6202) by @volokluev
• feat(eap): Add the TimeSeriesProcessor to entities/eap_spans (#6195) by @colin-sentry
• feat(EAP) Add Make directive that creates proper import paths (#6196) by @colin-sentry
• fix(capman): sets the correct throttle policy in query result metadata when multiple throttle policies present (#6192) by @xurui-c
• feat(generic-metrics): Bump materialization_version for generic metrics (#6194) by @john-z-yang
• feat: make deletes endpoint compatible with snuba SDK (#6193) by @kylemumma
• Change CODEOWNERS for EAP team (#6187) by @colin-sentry
• feat(generic-metrics): Forward sampling information from consumer to clickhouse (#6177) by @john-z-yang
• Increase the default throttle/warning threshold for allocation policies (#6189) by @xurui-c
• fix(eap): Use arrayElement directly to fix alias issues with HashBucketMapper (#6188) by @colin-sentry
• fix: Migrations ended up with the same number (#6183) by @evanh
• ref(snuba-deletes): introduce delete allocation policies (#6180) by @MeredithAnya
• feat(eap): Add an example endpoint that uses protobuf over http (#6173) by @colin-sentry
• feat(generic-metrics): Add migrations to allow sampling for distributions (#6172) by @enochtangg

Plus 40 more

24.7.1

Various fixes & improvements

• feat(cleanup): remove complicated readthrough cache (#6130) by @volokluev
• fix(transactions): Select transactions entity when using profiler id (#6133) by @Zylphrex
• Revert "Add processor for new spans schema (#6123)" (d60b3e6) by @getsentry-bot
• Revert "Define the models for the new Events Analytics Platform (#6126)" (b1376d6) by @getsentry-bot
• Add processor for new spans schema (#6123) by @colin-sentry
• Define the models for the new Events Analytics Platform (#6126) by @colin-sentry
• code (#6129) by @xurui-c
• fix: wait on system.mutations between hosts during migration execution (#6121) by @kylemumma
• feat(transactions): Make profiler queryable (#6122) by @Zylphrex
• bug(mql): Fix MQL totals queries (#6125) by @enochtangg
• chore: update some readmes related to migrations and autogen (#6076) by @kylemumma
• ref(lwdeletes): create parse_and_run_query function (#6120) by @volokluev
• feat(profiling): Write profiler_id to transactions table (#6111) by @Zylphrex
• Give telemetry experience team access to cardinality analyzer (#6115) by @vgrozdanic
• Add an inner join relationship for search_issues and group_attributes (#6119) by @snigdhas
• support max_bytes_before_external_group_by ClickHouse insert flag (#6118) by @onewland
• fix(mql): Nested filter conditions (#6117) by @john-z-yang
• code (#6114) by @xurui-c
• code (#6113) by @xurui-c
• chore: cleanup SEARCH_ISSUES_TMP storage set (#6110) by @kylemumma
• code (#6105) by @xurui-c
• chore: make snuba co-owners of datasets again (#6104) by @kylemumma
• fix: LowCardinality column bug w/ aliases (#6108) by @MeredithAnya
• meta: Bump new development version (5dc9763)

Plus 2 more

24.7.0

Various fixes & improvements

• feat(mql): MQL JOIN subquery generator (#6101) by @enochtangg
• feat(metrics): add wildcard support to MQL parser (#5972) by @shellmayr
• Revert "feat(profiling): Add profiler id column to transactions (#6099)" (f941b5e) by @getsentry-bot
• feat(profiling): Add profiler id column to transactions (#6099) by @Zylphrex
• feat: create search_issues_dist_v2 on same query node as group_attributes (#6087) by @kylemumma
• feat(replays): Set platform as empty string if platform is not provided (#6098) by @cmanallen
• feat(mql): Add test for formula queries with curried aggregate (#6080) by @john-z-yang
• feat(mql): Arbitrary nested column aggregates (#6096) by @john-z-yang
• dev(rate-limits): bump backfill query allocation to 30 per s (#6074) by @JoshFerge
• ref(snuba-deletes): Skeleton for DeletionProcessor (#6095) by @xurui-c
• ref(snuba-deletes): add DeletionSettings (#6092) by @MeredithAnya
• fix: move an import that may be causing issue with sentry devservice (#6090) by @kylemumma
• Revert "code (#6088)" (#6089) by @kneeyo1
• fix(capman): emit the bytes scanned metric from db_query (#6075) by @volokluev
• code (#6088) by @xurui-c
• feat(mql): Support groupby parsing for formula join queries (#6077) by @enochtangg
• Revert "code (#6081)" (ba3e5ed) by @getsentry-bot
• code (#6081) by @xurui-c
• Add clickhouse override settings (#6085) by @cmanallen
• chore: new temporary storage set (#6079) by @kylemumma
• feat(replays): Add materialization YAML definitions (#6071) by @cmanallen
• fix(devenv): Auto-install cargo watch (#6078) by @untitaker
• feat: autogenerate addcolumn migrations (#6053) by @kylemumma
• test(mql): Add tests for formula queries with scalar values (#6073) by @john-z-yang

Plus 29 more