Skip to content

Conversation

GrigoriyPA
Copy link
Collaborator

Changelog entry

Added new entity streaming query into scheme shard, also added this entry into cli / sdk (part of streaming queries support)

Changelog category

  • Not for changelog (changelog entry is not required)

Description for reviewers

@GrigoriyPA GrigoriyPA requested review from a team as code owners August 8, 2025 13:26
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for streaming queries as a new entity type in YDB's scheme shard. This is part of a larger effort to implement streaming query functionality within YDB.

Key changes include:

  • Added StreamingQuery as a new path type in the schema with full CRUD operations (Create, Read, Update, Delete)
  • Integrated streaming query support into CLI, SDK, and various system components
  • Added comprehensive test coverage for streaming query operations

Reviewed Changes

Copilot reviewed 53 out of 53 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
flat_schemeshard.schema Added StreamingQuery table definition to schema
ydb_backup_ut.cpp Added streaming query path type handling in backup tests
scheme.cpp/scheme.h Added StreamingQuery enum value and conversion logic
print_utils.cpp Added display formatting for streaming query entries
ydb_schema.cpp Added streaming query type mapping for completions
schema.cpp/kicli.h Added streaming query drop operation and enum support
ydb_scheme.proto Added STREAMING_QUERY enum value
core_ydb_impl.h Added streaming query to special case mappings
viewer.proto/browse.h Added streaming query viewer support
schemereq.cpp Added streaming query operation handling
ya.make Added streaming query source files and test directories
ut_system_names.cpp Added streaming query creation test case
Multiple schemeshard files Core implementation of streaming query operations
Multiple test files Comprehensive test coverage for streaming query functionality

Copy link

github-actions bot commented Aug 8, 2025

🟢 2025-08-08 13:29:37 UTC The validation of the Pull Request description is successful.

Copy link

github-actions bot commented Aug 8, 2025

2025-08-08 13:33:33 UTC Pre-commit check linux-x86_64-relwithdebinfo for 1f20f1f has started.
2025-08-08 13:33:47 UTC Artifacts will be uploaded here
2025-08-08 13:37:46 UTC ya make is running...
🟢 2025-08-08 15:41:45 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40140 37341 0 0 2760 39

🟢 2025-08-08 15:45:24 UTC Build successful.
🟡 2025-08-08 15:45:47 UTC ydbd size 2.2 GiB changed* by +920.2 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: f2c19c8 merge: 1f20f1f diff diff %
ydbd size 2 414 893 920 Bytes 2 415 836 256 Bytes +920.2 KiB +0.039%
ydbd stripped size 504 955 912 Bytes 505 059 240 Bytes +100.9 KiB +0.020%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 8, 2025

2025-08-08 13:35:19 UTC Pre-commit check linux-x86_64-release-asan for 1f20f1f has started.
2025-08-08 13:35:35 UTC Artifacts will be uploaded here
2025-08-08 13:39:28 UTC ya make is running...
🟡 2025-08-08 16:11:26 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
17497 17075 0 91 309 22

🟢 2025-08-08 16:12:57 UTC Build successful.
🟡 2025-08-08 16:13:23 UTC ydbd size 4.0 GiB changed* by +1.3 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: f2c19c8 merge: 1f20f1f diff diff %
ydbd size 4 248 576 568 Bytes 4 249 990 400 Bytes +1.3 MiB +0.033%
ydbd stripped size 1 470 134 040 Bytes 1 470 401 144 Bytes +260.8 KiB +0.018%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Gazizonoki
Gazizonoki previously approved these changes Aug 8, 2025
Copy link
Collaborator

@Gazizonoki Gazizonoki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SDK -ok

@Gazizonoki Gazizonoki requested a review from pnv1 August 8, 2025 14:18
kardymonds
kardymonds previously approved these changes Aug 11, 2025
@GrigoriyPA GrigoriyPA dismissed stale reviews from kardymonds and Gazizonoki via 9a19fe6 August 13, 2025 12:59
@GrigoriyPA GrigoriyPA force-pushed the YQ-4482-add-streaming-queries-into-SS branch from 0078f26 to 9a19fe6 Compare August 13, 2025 12:59
@GrigoriyPA GrigoriyPA requested a review from kardymonds August 13, 2025 12:59
Copy link

github-actions bot commented Aug 13, 2025

2025-08-13 13:00:35 UTC Pre-commit check linux-x86_64-relwithdebinfo for 7947976 has started.
2025-08-13 13:01:21 UTC Artifacts will be uploaded here
2025-08-13 13:05:51 UTC ya make is running...
🟡 2025-08-13 15:00:28 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40188 37419 0 15 2718 36

2025-08-13 15:04:10 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-08-13 15:19:35 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
284 (only retried tests) 259 0 3 0 22

2025-08-13 15:19:45 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-08-13 15:34:32 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
225 (only retried tests) 201 0 2 0 22

🟢 2025-08-13 15:34:40 UTC Build successful.
🟡 2025-08-13 15:35:01 UTC ydbd size 2.3 GiB changed* by +919.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: e4bcd1e merge: 7947976 diff diff %
ydbd size 2 425 456 376 Bytes 2 426 397 872 Bytes +919.4 KiB +0.039%
ydbd stripped size 506 646 344 Bytes 506 749 352 Bytes +100.6 KiB +0.020%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 13, 2025

2025-08-13 13:02:23 UTC Pre-commit check linux-x86_64-release-asan for 7947976 has started.
2025-08-13 13:02:37 UTC Artifacts will be uploaded here
2025-08-13 13:06:29 UTC ya make is running...
🟡 2025-08-13 15:28:42 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
17532 17136 0 126 248 22

🟢 2025-08-13 15:30:17 UTC Build successful.
🟡 2025-08-13 15:30:46 UTC ydbd size 4.0 GiB changed* by +1.3 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: e4bcd1e merge: 7947976 diff diff %
ydbd size 4 267 869 824 Bytes 4 269 283 360 Bytes +1.3 MiB +0.033%
ydbd stripped size 1 478 203 288 Bytes 1 478 470 424 Bytes +260.9 KiB +0.018%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@GrigoriyPA GrigoriyPA added the rebase-and-check Rebase PR with the current base branch and check label Aug 13, 2025
@github-actions github-actions bot removed the rebase-and-check Rebase PR with the current base branch and check label Aug 13, 2025
Copy link

github-actions bot commented Aug 13, 2025

2025-08-13 16:37:45 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9cc41df has started.
2025-08-13 16:37:59 UTC Artifacts will be uploaded here
2025-08-13 16:41:58 UTC ya make is running...
🟡 2025-08-13 18:40:28 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40192 37441 0 4 2710 37

2025-08-13 18:44:02 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-08-13 18:58:21 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
482 (only retried tests) 456 0 0 0 26

🟢 2025-08-13 18:58:30 UTC Build successful.
🟡 2025-08-13 18:58:48 UTC ydbd size 2.3 GiB changed* by +919.4 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 51cf589 merge: 9cc41df diff diff %
ydbd size 2 425 616 128 Bytes 2 426 557 560 Bytes +919.4 KiB +0.039%
ydbd stripped size 506 672 264 Bytes 506 775 208 Bytes +100.5 KiB +0.020%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 13, 2025

2025-08-13 16:37:55 UTC Pre-commit check linux-x86_64-release-asan for 9cc41df has started.
2025-08-13 16:38:12 UTC Artifacts will be uploaded here
2025-08-13 16:42:14 UTC ya make is running...
🟡 2025-08-13 19:13:40 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
17536 17138 0 122 253 23

🟢 2025-08-13 19:15:14 UTC Build successful.
🟡 2025-08-13 19:15:42 UTC ydbd size 4.0 GiB changed* by +1.3 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 51cf589 merge: 9cc41df diff diff %
ydbd size 4 268 147 352 Bytes 4 269 556 744 Bytes +1.3 MiB +0.033%
ydbd stripped size 1 478 297 496 Bytes 1 478 560 472 Bytes +256.8 KiB +0.018%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@GrigoriyPA GrigoriyPA force-pushed the YQ-4482-add-streaming-queries-into-SS branch from 9a19fe6 to 37de9fa Compare August 19, 2025 17:07
Copy link

github-actions bot commented Aug 19, 2025

2025-08-19 17:09:49 UTC Pre-commit check linux-x86_64-release-asan for b8b0728 has started.
2025-08-19 17:10:02 UTC Artifacts will be uploaded here
2025-08-19 17:14:08 UTC ya make is running...
🔴 2025-08-19 17:16:17 UTC Build failed, see the logs. Also see fail summary

@GrigoriyPA GrigoriyPA force-pushed the YQ-4482-add-streaming-queries-into-SS branch from 7901172 to c70d179 Compare August 29, 2025 19:26
Copy link

github-actions bot commented Aug 29, 2025

2025-08-29 19:28:30 UTC Pre-commit check linux-x86_64-release-asan for 58737e5 has started.
2025-08-29 19:28:34 UTC Artifacts will be uploaded here
2025-08-29 19:32:30 UTC ya make is running...
🟡 2025-08-29 22:15:12 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
17900 17448 0 117 303 32

🟢 2025-08-29 22:16:48 UTC Build successful.
🟡 2025-08-29 22:17:21 UTC ydbd size 4.0 GiB changed* by +1.3 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: bef2d26 merge: 58737e5 diff diff %
ydbd size 4 284 567 288 Bytes 4 285 973 792 Bytes +1.3 MiB +0.033%
ydbd stripped size 1 486 099 128 Bytes 1 486 361 592 Bytes +256.3 KiB +0.018%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Aug 29, 2025

2025-08-29 19:29:05 UTC Pre-commit check linux-x86_64-relwithdebinfo for 58737e5 has started.
2025-08-29 19:29:20 UTC Artifacts will be uploaded here
2025-08-29 19:33:19 UTC ya make is running...
🟡 2025-08-29 21:45:19 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40643 37767 0 2 2827 47

2025-08-29 21:48:49 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-08-29 22:03:19 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
735 (only retried tests) 710 0 0 0 25

🟢 2025-08-29 22:03:28 UTC Build successful.
🟡 2025-08-29 22:03:49 UTC ydbd size 2.3 GiB changed* by +922.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: bef2d26 merge: 58737e5 diff diff %
ydbd size 2 437 777 904 Bytes 2 438 722 184 Bytes +922.1 KiB +0.039%
ydbd stripped size 510 668 200 Bytes 510 771 592 Bytes +101.0 KiB +0.020%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 3, 2025

2025-09-03 08:59:15 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0f9c1ac has started.
2025-09-03 08:59:29 UTC Artifacts will be uploaded here
2025-09-03 09:03:31 UTC ya make is running...
🔴 2025-09-03 09:35:34 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Sep 3, 2025

2025-09-03 09:01:07 UTC Pre-commit check linux-x86_64-release-asan for 0f9c1ac has started.
2025-09-03 09:01:16 UTC Artifacts will be uploaded here
2025-09-03 09:05:17 UTC ya make is running...
🔴 2025-09-03 09:18:49 UTC Build failed, see the logs. Also see fail summary

@GrigoriyPA GrigoriyPA force-pushed the YQ-4482-add-streaming-queries-into-SS branch from c9472ea to 0f7b535 Compare September 3, 2025 12:28
Copy link

github-actions bot commented Sep 3, 2025

2025-09-03 12:35:53 UTC Pre-commit check linux-x86_64-release-asan for eb5ef40 has started.
2025-09-03 12:36:26 UTC Artifacts will be uploaded here
2025-09-03 12:41:35 UTC ya make is running...
🔴 2025-09-03 12:44:10 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Sep 3, 2025

2025-09-03 12:51:00 UTC Pre-commit check linux-x86_64-relwithdebinfo for eb5ef40 has started.
2025-09-03 12:51:26 UTC Artifacts will be uploaded here
2025-09-03 12:56:22 UTC ya make is running...
🔴 2025-09-03 12:58:22 UTC Build failed, see the logs. Also see fail summary

Copy link

github-actions bot commented Sep 3, 2025

2025-09-03 14:30:22 UTC Pre-commit check linux-x86_64-release-asan for 977159d has started.
2025-09-03 14:30:37 UTC Artifacts will be uploaded here
2025-09-03 14:34:36 UTC ya make is running...
2025-09-03 14:47:53 UTC Check cancelled

Copy link

github-actions bot commented Sep 3, 2025

2025-09-03 14:31:18 UTC Pre-commit check linux-x86_64-relwithdebinfo for 977159d has started.
2025-09-03 14:31:32 UTC Artifacts will be uploaded here
2025-09-03 14:35:34 UTC ya make is running...
2025-09-03 14:47:53 UTC Check cancelled

Copy link

github-actions bot commented Sep 3, 2025

2025-09-03 14:49:20 UTC Pre-commit check linux-x86_64-relwithdebinfo for 29c419f has started.
2025-09-03 14:49:36 UTC Artifacts will be uploaded here
2025-09-03 14:53:46 UTC ya make is running...
🟡 2025-09-03 16:56:12 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40777 38026 0 2 2716 33

2025-09-03 17:00:19 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-09-03 17:13:49 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
561 (only retried tests) 535 0 0 0 26

🟢 2025-09-03 17:13:58 UTC Build successful.
🟡 2025-09-03 17:14:15 UTC ydbd size 2.3 GiB changed* by +889.2 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 9bc3eb2 merge: 29c419f diff diff %
ydbd size 2 440 149 152 Bytes 2 441 059 648 Bytes +889.2 KiB +0.037%
ydbd stripped size 511 002 152 Bytes 511 102 472 Bytes +98.0 KiB +0.020%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 3, 2025

2025-09-03 14:49:21 UTC Pre-commit check linux-x86_64-release-asan for 29c419f has started.
2025-09-03 14:49:37 UTC Artifacts will be uploaded here
2025-09-03 14:53:52 UTC ya make is running...
🟡 2025-09-03 17:41:38 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
18018 17506 0 163 324 25

🟢 2025-09-03 17:43:13 UTC Build successful.
🟡 2025-09-03 17:43:37 UTC ydbd size 4.0 GiB changed* by +1.3 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 9bc3eb2 merge: 29c419f diff diff %
ydbd size 4 288 600 168 Bytes 4 289 982 176 Bytes +1.3 MiB +0.032%
ydbd stripped size 1 487 080 536 Bytes 1 487 351 192 Bytes +264.3 KiB +0.018%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 4, 2025

2025-09-04 07:22:42 UTC Pre-commit check linux-x86_64-relwithdebinfo for a0c49c1 has started.
2025-09-04 07:22:57 UTC Artifacts will be uploaded here
2025-09-04 07:27:09 UTC ya make is running...
2025-09-04 07:27:47 UTC Check cancelled

Copy link

github-actions bot commented Sep 4, 2025

2025-09-04 07:23:05 UTC Pre-commit check linux-x86_64-release-asan for a0c49c1 has started.
2025-09-04 07:23:19 UTC Artifacts will be uploaded here
2025-09-04 07:27:16 UTC ya make is running...
2025-09-04 07:27:47 UTC Check cancelled

Copy link

github-actions bot commented Sep 4, 2025

2025-09-04 07:41:11 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0b7e96c has started.
2025-09-04 07:41:16 UTC Artifacts will be uploaded here
2025-09-04 07:45:15 UTC ya make is running...
🟡 2025-09-04 09:40:16 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40779 38027 0 6 2715 31

2025-09-04 09:43:52 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-09-04 09:57:39 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
298 (only retried tests) 272 0 0 0 26

🟢 2025-09-04 09:57:48 UTC Build successful.
🟡 2025-09-04 09:58:05 UTC ydbd size 2.3 GiB changed* by +900.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 444f586 merge: 0b7e96c diff diff %
ydbd size 2 440 359 648 Bytes 2 441 282 048 Bytes +900.8 KiB +0.038%
ydbd stripped size 511 020 200 Bytes 511 126 216 Bytes +103.5 KiB +0.021%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 4, 2025

2025-09-04 07:41:40 UTC Pre-commit check linux-x86_64-release-asan for 0b7e96c has started.
2025-09-04 07:41:56 UTC Artifacts will be uploaded here
2025-09-04 07:45:54 UTC ya make is running...
🟡 2025-09-04 10:20:08 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
18019 17540 0 118 330 31

🟢 2025-09-04 10:21:43 UTC Build successful.
🟡 2025-09-04 10:22:05 UTC ydbd size 4.0 GiB changed* by +1.3 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 444f586 merge: 0b7e96c diff diff %
ydbd size 4 289 044 400 Bytes 4 290 441 720 Bytes +1.3 MiB +0.033%
ydbd stripped size 1 487 158 104 Bytes 1 487 434 456 Bytes +269.9 KiB +0.019%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@GrigoriyPA GrigoriyPA merged commit 34ff5ff into ydb-platform:main Sep 4, 2025
14 checks passed
@GrigoriyPA GrigoriyPA deleted the YQ-4482-add-streaming-queries-into-SS branch September 4, 2025 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants