Class: Rdkafka::Admin
- Inherits:
-
Object
- Object
- Rdkafka::Admin
- Includes:
- Helpers::Metadata, Helpers::OAuth
- Defined in:
- lib/rdkafka/admin.rb,
lib/rdkafka/admin/create_acl_handle.rb,
lib/rdkafka/admin/create_acl_report.rb,
lib/rdkafka/admin/delete_acl_handle.rb,
lib/rdkafka/admin/delete_acl_report.rb,
lib/rdkafka/admin/acl_binding_result.rb,
lib/rdkafka/admin/create_topic_handle.rb,
lib/rdkafka/admin/create_topic_report.rb,
lib/rdkafka/admin/delete_topic_handle.rb,
lib/rdkafka/admin/delete_topic_report.rb,
lib/rdkafka/admin/describe_acl_handle.rb,
lib/rdkafka/admin/describe_acl_report.rb,
lib/rdkafka/admin/list_offsets_handle.rb,
lib/rdkafka/admin/list_offsets_report.rb,
lib/rdkafka/admin/delete_groups_handle.rb,
lib/rdkafka/admin/delete_groups_report.rb,
lib/rdkafka/admin/config_binding_result.rb,
lib/rdkafka/admin/describe_configs_handle.rb,
lib/rdkafka/admin/describe_configs_report.rb,
lib/rdkafka/admin/create_partitions_handle.rb,
lib/rdkafka/admin/create_partitions_report.rb,
lib/rdkafka/admin/config_resource_binding_result.rb,
lib/rdkafka/admin/incremental_alter_configs_handle.rb,
lib/rdkafka/admin/incremental_alter_configs_report.rb
Overview
Admin client for Kafka administrative operations
Defined Under Namespace
Classes: AclBindingResult, ConfigBindingResult, ConfigResourceBindingResult, CreateAclHandle, CreateAclReport, CreatePartitionsHandle, CreatePartitionsReport, CreateTopicHandle, CreateTopicReport, DeleteAclHandle, DeleteAclReport, DeleteGroupsHandle, DeleteGroupsReport, DeleteTopicHandle, DeleteTopicReport, DescribeAclHandle, DescribeAclReport, DescribeConfigsHandle, DescribeConfigsReport, IncrementalAlterConfigsHandle, IncrementalAlterConfigsReport, ListOffsetsHandle, ListOffsetsReport
Class Method Summary collapse
-
.describe_errors ⇒ Hash{Integer => Hash}
Allows us to retrieve librdkafka errors with descriptions Useful for debugging and building UIs, etc.
Instance Method Summary collapse
-
#close ⇒ Object
Close this admin instance.
-
#closed? ⇒ Boolean
Whether this admin has closed.
-
#create_acl(resource_type:, resource_name:, resource_pattern_type:, principal:, host:, operation:, permission_type:) ⇒ CreateAclHandle
Create acl.
-
#create_partitions(topic_name, partition_count) ⇒ CreatePartitionsHandle
Creates extra partitions for a given topic.
-
#create_topic(topic_name, partition_count, replication_factor, topic_config = {}) ⇒ CreateTopicHandle
Create a topic with the given partition count and replication factor.
-
#delete_acl(resource_type:, resource_name:, resource_pattern_type:, principal:, host:, operation:, permission_type:) ⇒ DeleteAclHandle
Delete acl.
-
#delete_group(group_id) ⇒ DeleteGroupsHandle
Deletes a consumer group.
-
#delete_topic(topic_name) ⇒ DeleteTopicHandle
Deletes the named topic.
-
#describe_acl(resource_type:, resource_name:, resource_pattern_type:, principal:, host:, operation:, permission_type:) ⇒ DescribeAclHandle
Describe acls.
-
#describe_configs(resources) ⇒ DescribeConfigsHandle
Describe configs.
-
#enable_background_queue_io_events(fd, payload = "\x01") ⇒ nil
Enable IO event notifications for background events.
-
#enable_queue_io_events(fd, payload = "\x01") ⇒ nil
Enable IO event notifications for fiber scheduler integration When admin operations complete, librdkafka will write to your FD.
-
#events_poll_nb_each {|count| ... } ⇒ nil
Polls for events in a non-blocking loop, yielding the count after each iteration.
-
#incremental_alter_configs(resources_with_configs) ⇒ IncrementalAlterConfigsHandle
Alters in an incremental way all the configs provided for given resources.
-
#initialize(native_kafka) ⇒ Admin
constructor
A new instance of Admin.
-
#list_offsets(topic_partition_offsets, isolation_level: nil) ⇒ ListOffsetsHandle
Queries partition offsets by specification (earliest, latest, max_timestamp, or by timestamp) without requiring a consumer group.
-
#name ⇒ String
Admin name.
-
#start ⇒ Object
Starts the native Kafka polling thread and kicks off the init polling.
Methods included from Helpers::Metadata
Methods included from Helpers::OAuth
#oauthbearer_set_token, #oauthbearer_set_token_failure
Constructor Details
#initialize(native_kafka) ⇒ Admin
Returns a new instance of Admin.
55 56 57 58 59 60 |
# File 'lib/rdkafka/admin.rb', line 55 def initialize(native_kafka) @native_kafka = native_kafka # Makes sure, that native kafka gets closed before it gets GCed by Ruby ObjectSpace.define_finalizer(self, native_kafka.finalizer) end |
Class Method Details
.describe_errors ⇒ Hash{Integer => Hash}
Allows us to retrieve librdkafka errors with descriptions Useful for debugging and building UIs, etc.
14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
# File 'lib/rdkafka/admin.rb', line 14 def describe_errors # Memory pointers for the array of structures and count p_error_descs = FFI::MemoryPointer.new(:pointer) p_count = FFI::MemoryPointer.new(:size_t) # Call the attached function Bindings.rd_kafka_get_err_descs(p_error_descs, p_count) # Retrieve the number of items in the array count = p_count.read_uint # Get the pointer to the array of error descriptions array_of_errors = FFI::Pointer.new(Bindings::NativeErrorDesc, p_error_descs.read_pointer) errors = {} count.times do |i| # Get the pointer to each struct error_ptr = array_of_errors[i] # Create a new instance of NativeErrorDesc for each item error_desc = Bindings::NativeErrorDesc.new(error_ptr) # Read values from the struct code = error_desc[:code] name = "" desc = "" name = error_desc[:name].read_string unless error_desc[:name].null? desc = error_desc[:desc].read_string unless error_desc[:desc].null? errors[code] = { code: code, name: name, description: desc } end errors end |
Instance Method Details
#close ⇒ Object
Close this admin instance
135 136 137 138 139 |
# File 'lib/rdkafka/admin.rb', line 135 def close return if closed? ObjectSpace.undefine_finalizer(self) @native_kafka.close end |
#closed? ⇒ Boolean
Whether this admin has closed
142 143 144 |
# File 'lib/rdkafka/admin.rb', line 142 def closed? @native_kafka.closed? end |
#create_acl(resource_type:, resource_name:, resource_pattern_type:, principal:, host:, operation:, permission_type:) ⇒ CreateAclHandle
Create acl
438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 |
# File 'lib/rdkafka/admin.rb', line 438 def create_acl(resource_type:, resource_name:, resource_pattern_type:, principal:, host:, operation:, permission_type:) closed_admin_check(__method__) # Create a rd_kafka_AclBinding_t representing the new acl error_buffer = FFI::MemoryPointer.from_string(" " * 256) new_acl_ptr = Rdkafka::Bindings.rd_kafka_AclBinding_new( resource_type, FFI::MemoryPointer.from_string(resource_name), resource_pattern_type, FFI::MemoryPointer.from_string(principal), FFI::MemoryPointer.from_string(host), operation, , error_buffer, 256 ) if new_acl_ptr.null? raise Rdkafka::Config::ConfigError.new(error_buffer.read_string) end # Note that rd_kafka_CreateAcls can create more than one acl at a time pointer_array = [new_acl_ptr] acls_array_ptr = FFI::MemoryPointer.new(:pointer) acls_array_ptr.write_array_of_pointer(pointer_array) # Get a pointer to the queue that our request will be enqueued on queue_ptr = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_queue_get_background(inner) end if queue_ptr.null? Rdkafka::Bindings.rd_kafka_AclBinding_destroy(new_acl_ptr) raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end # Create and register the handle that we will return to the caller create_acl_handle = CreateAclHandle.new create_acl_handle[:pending] = true create_acl_handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA CreateAclHandle.register(create_acl_handle) = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_AdminOptions_new(inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_CREATEACLS) end Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, create_acl_handle.to_ptr) begin @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_CreateAcls( inner, acls_array_ptr, 1, , queue_ptr ) end rescue Exception CreateAclHandle.remove(create_acl_handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) Rdkafka::Bindings.rd_kafka_AclBinding_destroy(new_acl_ptr) end create_acl_handle end |
#create_partitions(topic_name, partition_count) ⇒ CreatePartitionsHandle
Creates extra partitions for a given topic
351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 |
# File 'lib/rdkafka/admin.rb', line 351 def create_partitions(topic_name, partition_count) closed_admin_check(__method__) @native_kafka.with_inner do |inner| error_buffer = FFI::MemoryPointer.from_string(" " * 256) new_partitions_ptr = Rdkafka::Bindings.rd_kafka_NewPartitions_new( FFI::MemoryPointer.from_string(topic_name), partition_count, error_buffer, 256 ) if new_partitions_ptr.null? raise Rdkafka::Config::ConfigError.new(error_buffer.read_string) end pointer_array = [new_partitions_ptr] topics_array_ptr = FFI::MemoryPointer.new(:pointer) topics_array_ptr.write_array_of_pointer(pointer_array) # Get a pointer to the queue that our request will be enqueued on queue_ptr = Rdkafka::Bindings.rd_kafka_queue_get_background(inner) if queue_ptr.null? Rdkafka::Bindings.rd_kafka_NewPartitions_destroy(new_partitions_ptr) raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end # Create and register the handle we will return to the caller create_partitions_handle = CreatePartitionsHandle.new create_partitions_handle[:pending] = true create_partitions_handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA CreatePartitionsHandle.register(create_partitions_handle) = Rdkafka::Bindings.rd_kafka_AdminOptions_new(inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_CREATEPARTITIONS) Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, create_partitions_handle.to_ptr) begin Rdkafka::Bindings.rd_kafka_CreatePartitions( inner, topics_array_ptr, 1, , queue_ptr ) rescue Exception CreatePartitionsHandle.remove(create_partitions_handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) Rdkafka::Bindings.rd_kafka_NewPartitions_destroy(new_partitions_ptr) end create_partitions_handle end end |
#create_topic(topic_name, partition_count, replication_factor, topic_config = {}) ⇒ CreateTopicHandle
Create a topic with the given partition count and replication factor
157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 |
# File 'lib/rdkafka/admin.rb', line 157 def create_topic(topic_name, partition_count, replication_factor, topic_config = {}) closed_admin_check(__method__) # Create a rd_kafka_NewTopic_t representing the new topic error_buffer = FFI::MemoryPointer.from_string(" " * 256) new_topic_ptr = Rdkafka::Bindings.rd_kafka_NewTopic_new( FFI::MemoryPointer.from_string(topic_name), partition_count, replication_factor, error_buffer, 256 ) if new_topic_ptr.null? raise Rdkafka::Config::ConfigError.new(error_buffer.read_string) end topic_config&.each do |key, value| Rdkafka::Bindings.rd_kafka_NewTopic_set_config( new_topic_ptr, key.to_s, value.to_s ) end # Note that rd_kafka_CreateTopics can create more than one topic at a time pointer_array = [new_topic_ptr] topics_array_ptr = FFI::MemoryPointer.new(:pointer) topics_array_ptr.write_array_of_pointer(pointer_array) # Get a pointer to the queue that our request will be enqueued on queue_ptr = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_queue_get_background(inner) end if queue_ptr.null? Rdkafka::Bindings.rd_kafka_NewTopic_destroy(new_topic_ptr) raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end # Create and register the handle we will return to the caller create_topic_handle = CreateTopicHandle.new create_topic_handle[:pending] = true create_topic_handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA CreateTopicHandle.register(create_topic_handle) = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_AdminOptions_new(inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_CREATETOPICS) end Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, create_topic_handle.to_ptr) begin @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_CreateTopics( inner, topics_array_ptr, 1, , queue_ptr ) end rescue Exception CreateTopicHandle.remove(create_topic_handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) Rdkafka::Bindings.rd_kafka_NewTopic_destroy(new_topic_ptr) end create_topic_handle end |
#delete_acl(resource_type:, resource_name:, resource_pattern_type:, principal:, host:, operation:, permission_type:) ⇒ DeleteAclHandle
Delete acl
539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 |
# File 'lib/rdkafka/admin.rb', line 539 def delete_acl(resource_type:, resource_name:, resource_pattern_type:, principal:, host:, operation:, permission_type:) closed_admin_check(__method__) # Create a rd_kafka_AclBinding_t representing the acl to be deleted error_buffer = FFI::MemoryPointer.from_string(" " * 256) delete_acl_ptr = Rdkafka::Bindings.rd_kafka_AclBindingFilter_new( resource_type, resource_name ? FFI::MemoryPointer.from_string(resource_name) : nil, resource_pattern_type, principal ? FFI::MemoryPointer.from_string(principal) : nil, host ? FFI::MemoryPointer.from_string(host) : nil, operation, , error_buffer, 256 ) if delete_acl_ptr.null? raise Rdkafka::Config::ConfigError.new(error_buffer.read_string) end # Note that rd_kafka_DeleteAcls can delete more than one acl at a time pointer_array = [delete_acl_ptr] acls_array_ptr = FFI::MemoryPointer.new(:pointer) acls_array_ptr.write_array_of_pointer(pointer_array) # Get a pointer to the queue that our request will be enqueued on queue_ptr = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_queue_get_background(inner) end if queue_ptr.null? Rdkafka::Bindings.rd_kafka_AclBinding_destroy(delete_acl_ptr) raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end # Create and register the handle that we will return to the caller delete_acl_handle = DeleteAclHandle.new delete_acl_handle[:pending] = true delete_acl_handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA DeleteAclHandle.register(delete_acl_handle) = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_AdminOptions_new(inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_DELETEACLS) end Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, delete_acl_handle.to_ptr) begin @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_DeleteAcls( inner, acls_array_ptr, 1, , queue_ptr ) end rescue Exception DeleteAclHandle.remove(delete_acl_handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) Rdkafka::Bindings.rd_kafka_AclBinding_destroy(delete_acl_ptr) end delete_acl_handle end |
#delete_group(group_id) ⇒ DeleteGroupsHandle
Deletes a consumer group
232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 |
# File 'lib/rdkafka/admin.rb', line 232 def delete_group(group_id) closed_admin_check(__method__) # Create a rd_kafka_DeleteGroup_t representing the new topic delete_groups_ptr = Rdkafka::Bindings.rd_kafka_DeleteGroup_new( FFI::MemoryPointer.from_string(group_id) ) pointer_array = [delete_groups_ptr] groups_array_ptr = FFI::MemoryPointer.new(:pointer) groups_array_ptr.write_array_of_pointer(pointer_array) # Get a pointer to the queue that our request will be enqueued on queue_ptr = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_queue_get_background(inner) end if queue_ptr.null? Rdkafka::Bindings.rd_kafka_DeleteGroup_destroy(delete_groups_ptr) raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end # Create and register the handle we will return to the caller delete_groups_handle = DeleteGroupsHandle.new delete_groups_handle[:pending] = true delete_groups_handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA DeleteGroupsHandle.register(delete_groups_handle) = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_AdminOptions_new(inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_DELETETOPICS) end Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, delete_groups_handle.to_ptr) begin @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_DeleteGroups( inner, groups_array_ptr, 1, , queue_ptr ) end rescue Exception DeleteGroupsHandle.remove(delete_groups_handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) Rdkafka::Bindings.rd_kafka_DeleteGroup_destroy(delete_groups_ptr) end delete_groups_handle end |
#delete_topic(topic_name) ⇒ DeleteTopicHandle
Deletes the named topic
291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 |
# File 'lib/rdkafka/admin.rb', line 291 def delete_topic(topic_name) closed_admin_check(__method__) # Create a rd_kafka_DeleteTopic_t representing the topic to be deleted delete_topic_ptr = Rdkafka::Bindings.rd_kafka_DeleteTopic_new(FFI::MemoryPointer.from_string(topic_name)) # Note that rd_kafka_DeleteTopics can create more than one topic at a time pointer_array = [delete_topic_ptr] topics_array_ptr = FFI::MemoryPointer.new(:pointer) topics_array_ptr.write_array_of_pointer(pointer_array) # Get a pointer to the queue that our request will be enqueued on queue_ptr = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_queue_get_background(inner) end if queue_ptr.null? Rdkafka::Bindings.rd_kafka_DeleteTopic_destroy(delete_topic_ptr) raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end # Create and register the handle we will return to the caller delete_topic_handle = DeleteTopicHandle.new delete_topic_handle[:pending] = true delete_topic_handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA DeleteTopicHandle.register(delete_topic_handle) = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_AdminOptions_new(inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_DELETETOPICS) end Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, delete_topic_handle.to_ptr) begin @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_DeleteTopics( inner, topics_array_ptr, 1, , queue_ptr ) end rescue Exception DeleteTopicHandle.remove(delete_topic_handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) Rdkafka::Bindings.rd_kafka_DeleteTopic_destroy(delete_topic_ptr) end delete_topic_handle end |
#describe_acl(resource_type:, resource_name:, resource_pattern_type:, principal:, host:, operation:, permission_type:) ⇒ DescribeAclHandle
Describe acls
642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 |
# File 'lib/rdkafka/admin.rb', line 642 def describe_acl(resource_type:, resource_name:, resource_pattern_type:, principal:, host:, operation:, permission_type:) closed_admin_check(__method__) # Create a rd_kafka_AclBinding_t with the filters to fetch existing acls error_buffer = FFI::MemoryPointer.from_string(" " * 256) describe_acl_ptr = Rdkafka::Bindings.rd_kafka_AclBindingFilter_new( resource_type, resource_name ? FFI::MemoryPointer.from_string(resource_name) : nil, resource_pattern_type, principal ? FFI::MemoryPointer.from_string(principal) : nil, host ? FFI::MemoryPointer.from_string(host) : nil, operation, , error_buffer, 256 ) if describe_acl_ptr.null? raise Rdkafka::Config::ConfigError.new(error_buffer.read_string) end # Get a pointer to the queue that our request will be enqueued on queue_ptr = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_queue_get_background(inner) end if queue_ptr.null? Rdkafka::Bindings.rd_kafka_AclBinding_destroy(describe_acl_ptr) raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end # Create and register the handle that we will return to the caller describe_acl_handle = DescribeAclHandle.new describe_acl_handle[:pending] = true describe_acl_handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA DescribeAclHandle.register(describe_acl_handle) = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_AdminOptions_new(inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_DESCRIBEACLS) end Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, describe_acl_handle.to_ptr) begin @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_DescribeAcls( inner, describe_acl_ptr, , queue_ptr ) end rescue Exception DescribeAclHandle.remove(describe_acl_handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) Rdkafka::Bindings.rd_kafka_AclBinding_destroy(describe_acl_ptr) end describe_acl_handle end |
#describe_configs(resources) ⇒ DescribeConfigsHandle
Several resources can be requested at one go, but only one broker at a time
Describe configs
716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 |
# File 'lib/rdkafka/admin.rb', line 716 def describe_configs(resources) closed_admin_check(__method__) # Parse user input before allocating any native resources or registering the handle, so a # missing :resource_type / :resource_name key raises with nothing to clean up. Previously the # KeyError leaked the background queue, the AdminOptions, the registered handle and any # ConfigResources already built. parsed_resources = resources.map do |resource_details| [resource_details.fetch(:resource_type), resource_details.fetch(:resource_name)] end handle = DescribeConfigsHandle.new handle[:pending] = true handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA queue_ptr = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_queue_get_background(inner) end if queue_ptr.null? raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_AdminOptions_new( inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_DESCRIBECONFIGS ) end DescribeConfigsHandle.register(handle) Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, handle.to_ptr) pointer_array = parsed_resources.map do |resource_type, resource_name| Rdkafka::Bindings.rd_kafka_ConfigResource_new( resource_type, FFI::MemoryPointer.from_string(resource_name) ) end configs_array_ptr = FFI::MemoryPointer.new(:pointer, pointer_array.size) configs_array_ptr.write_array_of_pointer(pointer_array) begin @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_DescribeConfigs( inner, configs_array_ptr, pointer_array.size, , queue_ptr ) end rescue Exception DescribeConfigsHandle.remove(handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) if configs_array_ptr Rdkafka::Bindings.rd_kafka_ConfigResource_destroy_array( configs_array_ptr, pointer_array.size ) end end handle end |
#enable_background_queue_io_events(fd, payload = "\x01") ⇒ nil
Enable IO event notifications for background events
91 92 93 |
# File 'lib/rdkafka/admin.rb', line 91 def enable_background_queue_io_events(fd, payload = "\x01") @native_kafka.enable_background_queue_io_events(fd, payload) end |
#enable_queue_io_events(fd, payload = "\x01") ⇒ nil
Enable IO event notifications for fiber scheduler integration When admin operations complete, librdkafka will write to your FD
82 83 84 |
# File 'lib/rdkafka/admin.rb', line 82 def enable_queue_io_events(fd, payload = "\x01") @native_kafka.enable_main_queue_io_events(fd, payload) end |
#events_poll_nb_each {|count| ... } ⇒ nil
This method holds the inner lock until the queue is empty or :stop is returned.
Other admin operations will wait until this method returns.
This method is thread-safe as it uses @native_kafka.with_inner synchronization
Polls for events in a non-blocking loop, yielding the count after each iteration.
This method processes events (stats, errors, etc.) in a single GVL/mutex session, which is more efficient than repeated individual polls. It uses non-blocking polls internally (no GVL release between polls).
Yields the count of events processed after each poll iteration, allowing the caller
to implement timeout or other termination logic by returning :stop.
122 123 124 125 126 127 128 129 130 131 132 |
# File 'lib/rdkafka/admin.rb', line 122 def events_poll_nb_each closed_admin_check(__method__) @native_kafka.with_inner do |inner| loop do count = Rdkafka::Bindings.rd_kafka_poll_nb(inner, 0) break if count.zero? break if yield(count) == :stop end end end |
#incremental_alter_configs(resources_with_configs) ⇒ IncrementalAlterConfigsHandle
Several resources can be requested at one go, but only one broker at a time
The results won't contain altered values but only the altered resources
Alters in an incremental way all the configs provided for given resources
name, value and the proper op_type to perform on this value.
800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 |
# File 'lib/rdkafka/admin.rb', line 800 def incremental_alter_configs(resources_with_configs) closed_admin_check(__method__) # Parse user input before allocating native resources or registering the handle, so a missing # key raises with nothing to clean up. Previously the KeyError leaked the background queue, # the AdminOptions, the registered handle and any ConfigResources already built. parsed_resources = resources_with_configs.map do |resource_details| [ resource_details.fetch(:resource_type), resource_details.fetch(:resource_name), resource_details.fetch(:configs).map do |config| [config.fetch(:name), config.fetch(:op_type), config.fetch(:value)] end ] end handle = IncrementalAlterConfigsHandle.new handle[:pending] = true handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA queue_ptr = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_queue_get_background(inner) end if queue_ptr.null? raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_AdminOptions_new( inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_INCREMENTALALTERCONFIGS ) end IncrementalAlterConfigsHandle.register(handle) Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, handle.to_ptr) add_error = nil pointer_array = parsed_resources.map do |resource_type, resource_name, configs| # First build the appropriate resource representation resource_ptr = Rdkafka::Bindings.rd_kafka_ConfigResource_new( resource_type, FFI::MemoryPointer.from_string(resource_name) ) configs.each do |name, op_type, value| # rd_kafka_ConfigResource_add_incremental_config returns a non-NULL rd_kafka_error_t for # an invalid op_type, an empty/nil name, or a nil value on a non-delete op. The result # used to be ignored: the entry was silently dropped, the alter request still reported # success, and the error object leaked. Capture the first error (always destroying the # native error object) and raise it below, before the request is sent. error_ptr = Bindings.rd_kafka_ConfigResource_add_incremental_config( resource_ptr, name, op_type, value ) unless error_ptr.null? code = Rdkafka::Bindings.rd_kafka_error_code(error_ptr) Rdkafka::Bindings.rd_kafka_error_destroy(error_ptr) unless code == Rdkafka::Bindings::RD_KAFKA_RESP_ERR_NO_ERROR add_error ||= Rdkafka::RdkafkaError.new( code, "rd_kafka_ConfigResource_add_incremental_config" ) end end end resource_ptr end configs_array_ptr = FFI::MemoryPointer.new(:pointer, pointer_array.size) configs_array_ptr.write_array_of_pointer(pointer_array) begin # Raise only after the full array is built so the ensure below frees every ConfigResource # we created, and before the request is sent so a rejected entry can't report success. raise add_error if add_error @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_IncrementalAlterConfigs( inner, configs_array_ptr, pointer_array.size, , queue_ptr ) end rescue Exception IncrementalAlterConfigsHandle.remove(handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) if configs_array_ptr Rdkafka::Bindings.rd_kafka_ConfigResource_destroy_array( configs_array_ptr, pointer_array.size ) end end handle end |
#list_offsets(topic_partition_offsets, isolation_level: nil) ⇒ ListOffsetsHandle
Queries partition offsets by specification (earliest, latest, max_timestamp, or by timestamp) without requiring a consumer group.
939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 |
# File 'lib/rdkafka/admin.rb', line 939 def list_offsets(topic_partition_offsets, isolation_level: nil) closed_admin_check(__method__) # Parse and validate every offset spec before allocating the native list, so a missing key or # an unknown offset specification raises with nothing to clean up. Previously the # ArgumentError (or KeyError) was raised after `rd_kafka_topic_partition_list_new`, leaking # the native list. parsed = topic_partition_offsets.flat_map do |topic, partitions| partitions.map do |spec| offset = spec.fetch(:offset) native_offset = case offset when :earliest then Rdkafka::Bindings::RD_KAFKA_OFFSET_SPEC_EARLIEST when :latest then Rdkafka::Bindings::RD_KAFKA_OFFSET_SPEC_LATEST when :max_timestamp then Rdkafka::Bindings::RD_KAFKA_OFFSET_SPEC_MAX_TIMESTAMP when Integer then offset else raise ArgumentError, "Unknown offset specification: #{offset.inspect}" end [topic, spec.fetch(:partition), native_offset] end end # Build native topic partition list tpl = Rdkafka::Bindings.rd_kafka_topic_partition_list_new(parsed.size) parsed.each do |topic, partition, native_offset| Rdkafka::Bindings.rd_kafka_topic_partition_list_add(tpl, topic, partition) Rdkafka::Bindings.rd_kafka_topic_partition_list_set_offset(tpl, topic, partition, native_offset) end # Get a pointer to the queue that our request will be enqueued on queue_ptr = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_queue_get_background(inner) end if queue_ptr.null? Rdkafka::Bindings.rd_kafka_topic_partition_list_destroy(tpl) raise Rdkafka::Config::ConfigError.new("rd_kafka_queue_get_background was NULL") end # Create and register the handle we will return to the caller handle = ListOffsetsHandle.new handle[:pending] = true handle[:response] = Rdkafka::Bindings::RD_KAFKA_PARTITION_UA = @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_AdminOptions_new( inner, Rdkafka::Bindings::RD_KAFKA_ADMIN_OP_LISTOFFSETS ) end if isolation_level Rdkafka::Bindings.rd_kafka_AdminOptions_set_isolation_level(, isolation_level) end ListOffsetsHandle.register(handle) Rdkafka::Bindings.rd_kafka_AdminOptions_set_opaque(, handle.to_ptr) begin @native_kafka.with_inner do |inner| Rdkafka::Bindings.rd_kafka_ListOffsets( inner, tpl, , queue_ptr ) end rescue Exception ListOffsetsHandle.remove(handle.to_ptr.address) raise ensure Rdkafka::Bindings.rd_kafka_AdminOptions_destroy() Rdkafka::Bindings.rd_kafka_queue_destroy(queue_ptr) Rdkafka::Bindings.rd_kafka_topic_partition_list_destroy(tpl) end handle end |
#name ⇒ String
Returns admin name.
69 70 71 72 73 |
# File 'lib/rdkafka/admin.rb', line 69 def name @name ||= @native_kafka.with_inner do |inner| ::Rdkafka::Bindings.rd_kafka_name(inner) end end |
#start ⇒ Object
Not needed to run unless explicit start was disabled
Starts the native Kafka polling thread and kicks off the init polling
64 65 66 |
# File 'lib/rdkafka/admin.rb', line 64 def start @native_kafka.start end |